2024-02-14T17:27:46.889 INFO:root:teuthology version: 0.0.1.dev258+g9e44c16 2024-02-14T17:27:46.889 DEBUG:teuthology.run:Teuthology command: teuthology --name gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi --owner scheduled_gabrioux@teuthology --description orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} --archive /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815 --verbose -- /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815/orig.config.yaml 2024-02-14T17:27:46.918 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T17:27:46.984 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815 branch: wip-guits-testing-2024-02-14-1033 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7559815' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-2024-02-14-1033 ceph: conf: client: debug ms: 1 global: mon election default strategy: 3 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-2024-02-14-1033 sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4718 sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-2024-02-14-1033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8727ce939fc044a98f215e29696bb1433f5758d1/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEqHc+axRfi/36fqFGGPS8jxmYiw1/UCCGtjBUSvJqo13qD9za4uRqHlUHPIR4RWqKOA74POHXdpLiQSSsIra4= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEFGDxPeEqtQJ9xNWG6du0Xh67LzpgHU8T1ihKTxxuqMTYU/2lVxGA3qH4l29cZMSeA0WXqZ75rdt9rTVkCwL78= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9e44c16382dfc4c612f7437cbb344c76223137cc timestamp: 2024-02-14_13:54:10 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-14T17:27:46.984 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8727ce939fc044a98f215e29696bb1433f5758d1/qa; will attempt to use it 2024-02-14T17:27:46.984 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8727ce939fc044a98f215e29696bb1433f5758d1/qa/tasks 2024-02-14T17:27:46.984 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-14T17:27:46.986 INFO:teuthology.task.internal:Checking packages... 2024-02-14T17:27:47.009 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8727ce939fc044a98f215e29696bb1433f5758d1' 2024-02-14T17:27:47.009 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-14T17:27:47.009 INFO:teuthology.packaging:ref: None 2024-02-14T17:27:47.009 INFO:teuthology.packaging:tag: None 2024-02-14T17:27:47.009 INFO:teuthology.packaging:branch: wip-guits-testing-2024-02-14-1033 2024-02-14T17:27:47.009 INFO:teuthology.packaging:sha1: 8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:27:47.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-2024-02-14-1033 2024-02-14T17:27:47.338 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1430.g8727ce93 2024-02-14T17:27:47.339 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-14T17:27:47.346 INFO:teuthology.task.internal:no buildpackages task found 2024-02-14T17:27:47.346 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-14T17:27:47.376 INFO:teuthology.task.internal:Saving configuration 2024-02-14T17:27:47.387 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-14T17:27:47.392 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T17:27:47.418 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 17:22:38.396018', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEqHc+axRfi/36fqFGGPS8jxmYiw1/UCCGtjBUSvJqo13qD9za4uRqHlUHPIR4RWqKOA74POHXdpLiQSSsIra4='} 2024-02-14T17:27:47.441 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 17:22:38.397865', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEFGDxPeEqtQJ9xNWG6du0Xh67LzpgHU8T1ihKTxxuqMTYU/2lVxGA3qH4l29cZMSeA0WXqZ75rdt9rTVkCwL78='} 2024-02-14T17:27:47.441 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-14T17:27:47.447 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-14T17:27:47.448 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-14T17:27:47.448 INFO:teuthology.run_tasks:Running task console_log... 2024-02-14T17:27:47.537 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc442e4e0d0>, signals=[15]) 2024-02-14T17:27:47.538 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-14T17:27:47.544 INFO:teuthology.task.internal:Opening connections... 2024-02-14T17:27:47.544 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-02-14T17:27:47.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:27:47.634 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-02-14T17:27:47.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:27:47.724 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-14T17:27:47.731 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-02-14T17:27:47.748 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-02-14T17:27:47.749 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-02-14T17:27:47.806 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-02-14T17:27:47.806 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-02-14T17:27:47.806 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-02-14T17:27:47.806 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-02-14T17:27:47.806 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-02-14T17:27:47.806 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-02-14T17:27:47.806 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-14T17:27:47.806 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-02-14T17:27:47.806 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-02-14T17:27:47.806 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-14T17:27:47.807 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-02-14T17:27:47.807 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T17:27:47.807 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-14T17:27:47.807 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T17:27:47.807 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-02-14T17:27:47.830 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-02-14T17:27:47.847 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-02-14T17:27:47.847 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="9" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="9" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el9" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:LOGO="fedora-logo-icon" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2024-02-14T17:27:47.907 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T17:27:47.908 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-14T17:27:47.908 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T17:27:47.908 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-02-14T17:27:47.930 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-14T17:27:47.941 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-14T17:27:47.959 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-14T17:27:47.960 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-02-14T17:27:47.962 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-02-14T17:27:47.977 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-14T17:27:47.983 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-14T17:27:47.983 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-02-14T17:27:48.018 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-02-14T17:27:48.125 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-14T17:27:48.180 INFO:teuthology.run_tasks:Running task kernel... 2024-02-14T17:27:48.194 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-14T17:27:48.194 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-14T17:27:48.194 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-14T17:27:48.195 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-14T17:27:48.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:27:48.210 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-02-14T17:27:48.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:27:48.211 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-02-14T17:27:48.268 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-361.el9.x86_64 2024-02-14T17:27:48.269 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-361.el9.x86_64 2024-02-14T17:27:48.269 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2024-02-14T17:27:48.270 INFO:teuthology.orchestra.run.smithi179.stdout:5.14.0-361.el9.x86_64 2024-02-14T17:27:48.270 INFO:teuthology.task.kernel:Running kernel on smithi179: 5.14.0-361.el9.x86_64 2024-02-14T17:27:48.270 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum install -y kernel 2024-02-14T17:27:52.017 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 3.2 MB/s | 6.7 MB 00:02 2024-02-14T17:27:52.095 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - AppStream 3.3 MB/s | 6.7 MB 00:02 2024-02-14T17:27:53.452 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.0 MB 00:00 2024-02-14T17:27:53.666 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-02-14T17:27:55.260 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-02-14T17:27:56.148 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.0 MB 00:02 2024-02-14T17:27:58.286 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 19 MB 00:01 2024-02-14T17:28:00.316 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 6.2 MB/s | 19 MB 00:03 2024-02-14T17:28:03.370 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.1 MB 00:01 2024-02-14T17:28:04.846 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 15 kB 00:00 2024-02-14T17:28:05.216 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.1 MB 00:01 2024-02-14T17:28:06.551 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-02-14T17:28:07.026 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 15 kB 00:00 2024-02-14T17:28:08.771 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-02-14T17:28:11.473 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-14T17:28:13.677 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-02-14T17:28:13.805 INFO:teuthology.orchestra.run.smithi179.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-14T17:28:13.806 INFO:teuthology.orchestra.run.smithi179.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-14T17:28:13.900 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:28:13.904 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout: kernel x86_64 5.14.0-420.el9 CentOS-BaseOS 6.0 M 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-core x86_64 5.14.0-420.el9 CentOS-BaseOS 21 M 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules x86_64 5.14.0-420.el9 CentOS-BaseOS 39 M 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-core x86_64 5.14.0-420.el9 CentOS-BaseOS 33 M 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout:Install 4 Packages 2024-02-14T17:28:13.905 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:28:13.906 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 99 M 2024-02-14T17:28:13.906 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 122 M 2024-02-14T17:28:13.906 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:28:14.341 INFO:teuthology.orchestra.run.smithi179.stdout:(1/4): kernel-5.14.0-420.el9.x86_64.rpm 14 MB/s | 6.0 MB 00:00 2024-02-14T17:28:14.983 INFO:teuthology.orchestra.run.smithi179.stdout:(2/4): kernel-core-5.14.0-420.el9.x86_64.rpm 19 MB/s | 21 MB 00:01 2024-02-14T17:28:15.867 INFO:teuthology.orchestra.run.smithi179.stdout:(3/4): kernel-modules-5.14.0-420.el9.x86_64.rpm 20 MB/s | 39 MB 00:01 2024-02-14T17:28:15.950 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-14T17:28:15.950 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-14T17:28:16.064 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:28:16.068 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:28:16.068 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:28:16.068 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:28:16.068 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:28:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-420.el9 CentOS-BaseOS 6.0 M 2024-02-14T17:28:16.069 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-14T17:28:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 5.14.0-420.el9 CentOS-BaseOS 21 M 2024-02-14T17:28:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 5.14.0-420.el9 CentOS-BaseOS 39 M 2024-02-14T17:28:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core x86_64 5.14.0-420.el9 CentOS-BaseOS 33 M 2024-02-14T17:28:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:28:16.069 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:28:16.069 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:28:16.069 INFO:teuthology.orchestra.run.smithi100.stdout:Install 4 Packages 2024-02-14T17:28:16.069 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:28:16.070 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 99 M 2024-02-14T17:28:16.070 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 122 M 2024-02-14T17:28:16.070 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:28:16.250 INFO:teuthology.orchestra.run.smithi179.stdout:(4/4): kernel-modules-core-5.14.0-420.el9.x86_6 18 MB/s | 33 MB 00:01 2024-02-14T17:28:16.253 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:28:16.254 INFO:teuthology.orchestra.run.smithi179.stdout:Total 42 MB/s | 99 MB 00:02 2024-02-14T17:28:16.254 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:28:16.520 INFO:teuthology.orchestra.run.smithi100.stdout:(1/4): kernel-5.14.0-420.el9.x86_64.rpm 13 MB/s | 6.0 MB 00:00 2024-02-14T17:28:16.636 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:28:16.636 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:28:17.645 INFO:teuthology.orchestra.run.smithi100.stdout:(2/4): kernel-modules-5.14.0-420.el9.x86_64.rpm 25 MB/s | 39 MB 00:01 2024-02-14T17:28:17.664 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:28:17.665 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:28:17.854 INFO:teuthology.orchestra.run.smithi100.stdout:(3/4): kernel-core-5.14.0-420.el9.x86_64.rpm 12 MB/s | 21 MB 00:01 2024-02-14T17:28:18.270 INFO:teuthology.orchestra.run.smithi100.stdout:(4/4): kernel-modules-core-5.14.0-420.el9.x86_6 19 MB/s | 33 MB 00:01 2024-02-14T17:28:18.273 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:28:18.274 INFO:teuthology.orchestra.run.smithi100.stdout:Total 45 MB/s | 99 MB 00:02 2024-02-14T17:28:18.274 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:28:18.637 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:28:18.637 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:28:19.669 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:28:19.669 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:28:20.085 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:28:20.749 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-modules-core-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:28:20.879 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:28:22.134 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:28:22.991 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-core-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:28:23.138 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:28:23.624 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:28:23.813 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:28:25.914 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:28:26.117 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:28:28.534 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:28:28.751 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:28:30.853 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:28:31.064 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:28:33.443 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:28:35.832 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:29:02.847 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:29:02.848 INFO:teuthology.orchestra.run.smithi179.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-14T17:29:02.848 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:29:02.919 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:29:03.391 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:29:03.391 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:29:03.392 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:29:03.392 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:29:03.821 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:29:03.821 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:29:03.821 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:29:03.821 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-5.14.0-420.el9.x86_64 2024-02-14T17:29:03.821 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-core-5.14.0-420.el9.x86_64 2024-02-14T17:29:03.821 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-5.14.0-420.el9.x86_64 2024-02-14T17:29:03.821 INFO:teuthology.orchestra.run.smithi179.stdout: kernel-modules-core-5.14.0-420.el9.x86_64 2024-02-14T17:29:03.821 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:29:03.821 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:29:04.378 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:29:04.503 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:29:04.503 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:29:04.503 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-420.el9.x86_64 2024-02-14T17:29:04.503 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-14T17:29:04.504 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-14T17:29:04.504 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-14T17:29:04.504 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi179.front.sepia.ceph.com, path=None, version=distro) 2024-02-14T17:29:04.504 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:29:04.611 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:29:04.611 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:29:04.612 DEBUG:teuthology.orchestra.run.smithi179:> sudo rpm -qi grub2-tools 2024-02-14T17:29:04.654 INFO:teuthology.orchestra.run.smithi179.stdout:Name : grub2-tools 2024-02-14T17:29:04.654 INFO:teuthology.orchestra.run.smithi179.stdout:Epoch : 1 2024-02-14T17:29:04.654 INFO:teuthology.orchestra.run.smithi179.stdout:Version : 2.06 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Release : 68.el9 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Architecture: x86_64 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Group : Unspecified 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Size : 8264254 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:License : GPLv3+ 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Packager : builder@centos.org 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Vendor : CentOS 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-14T17:29:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:Summary : Support tools for GRUB. 2024-02-14T17:29:04.656 INFO:teuthology.orchestra.run.smithi179.stdout:Description : 2024-02-14T17:29:04.656 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:29:04.656 INFO:teuthology.orchestra.run.smithi179.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-14T17:29:04.656 INFO:teuthology.orchestra.run.smithi179.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-14T17:29:04.656 INFO:teuthology.orchestra.run.smithi179.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-14T17:29:04.656 INFO:teuthology.orchestra.run.smithi179.stdout:hardware devices. 2024-02-14T17:29:04.656 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:29:04.656 INFO:teuthology.orchestra.run.smithi179.stdout:This subpackage provides tools for support of all platforms. 2024-02-14T17:29:04.657 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-14T17:29:04.657 INFO:teuthology.task.kernel:Updating grub on smithi179 to boot 5.14.0-420.el9.x86_64 2024-02-14T17:29:04.657 DEBUG:teuthology.orchestra.run.smithi179:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-14T17:29:05.264 INFO:teuthology.orchestra.run.smithi179.stderr:Generating grub configuration file ... 2024-02-14T17:29:05.382 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:29:05.383 INFO:teuthology.orchestra.run.smithi100.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-14T17:29:05.383 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:29:05.452 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-420.el9.x86_64 1/4 2024-02-14T17:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-5.14.0-420.el9.x86_64 2/4 2024-02-14T17:29:05.864 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-5.14.0-420.el9.x86_64 3/4 2024-02-14T17:29:06.292 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-core-5.14.0-420.el9.x86_64 4/4 2024-02-14T17:29:06.292 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:29:06.292 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:29:06.292 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-420.el9.x86_64 2024-02-14T17:29:06.292 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core-5.14.0-420.el9.x86_64 2024-02-14T17:29:06.293 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-5.14.0-420.el9.x86_64 2024-02-14T17:29:06.293 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core-5.14.0-420.el9.x86_64 2024-02-14T17:29:06.293 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:29:06.293 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:29:06.655 INFO:teuthology.orchestra.run.smithi179.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-14T17:29:06.698 INFO:teuthology.orchestra.run.smithi179.stderr:done 2024-02-14T17:29:06.700 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-02-14T17:29:06.716 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.PDpVBQrb0Q 2024-02-14T17:29:06.716 DEBUG:teuthology.orchestra.run.smithi179:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PDpVBQrb0Q 2024-02-14T17:29:06.783 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.PDpVBQrb0Q 2024-02-14T17:29:06.790 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:29:06.924 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:29:06.925 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:29:06.925 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-420.el9.x86_64 2024-02-14T17:29:06.925 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-14T17:29:06.925 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-14T17:29:06.925 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-14T17:29:06.925 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-02-14T17:29:06.925 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-02-14T17:29:06.929 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.PDpVBQrb0Q is 6KB 2024-02-14T17:29:06.977 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.PDpVBQrb0Q 2024-02-14T17:29:06.991 DEBUG:teuthology.orchestra.run.smithi179:> sudo /bin/ls /boot/loader/entries || true 2024-02-14T17:29:07.033 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-420.el9.x86_64 2024-02-14T17:29:07.033 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-420.el9.x86_64 2024-02-14T17:29:07.034 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-02-14T17:29:07.057 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-14T17:29:07.057 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-14T17:29:07.057 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-14T17:29:07.057 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64.conf 2024-02-14T17:29:07.059 DEBUG:teuthology.orchestra.run.smithi179:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64 2024-02-14T17:29:07.068 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-02-14T17:29:07.068 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-02-14T17:29:07.068 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-02-14T17:29:07.068 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 68.el9 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8264254 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:29:07.069 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-14T17:29:07.070 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-14T17:29:07.070 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-14T17:29:07.070 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-02-14T17:29:07.070 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:29:07.070 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-02-14T17:29:07.071 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-14T17:29:07.071 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 5.14.0-420.el9.x86_64 2024-02-14T17:29:07.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-14T17:29:07.218 DEBUG:teuthology.orchestra.run.smithi179:> sudo shutdown -r now 2024-02-14T17:29:07.682 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-02-14T17:29:09.055 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-14T17:29:09.097 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-02-14T17:29:09.099 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-02-14T17:29:09.114 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.NSUokCHfI8 2024-02-14T17:29:09.115 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NSUokCHfI8 2024-02-14T17:29:09.184 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.NSUokCHfI8 2024-02-14T17:29:09.314 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.NSUokCHfI8 is 6KB 2024-02-14T17:29:09.362 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.NSUokCHfI8 2024-02-14T17:29:09.376 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-02-14T17:29:09.442 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-14T17:29:09.442 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-14T17:29:09.442 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-14T17:29:09.443 INFO:teuthology.orchestra.run.smithi100.stdout:15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64.conf 2024-02-14T17:29:09.444 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-420.el9.x86_64 2024-02-14T17:29:09.592 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-02-14T17:29:37.248 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-14T17:29:37.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-02-14T17:29:37.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:29:39.596 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-14T17:29:39.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-02-14T17:29:39.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:29:55.772 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-02-14T17:29:58.076 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-02-14T17:30:04.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-02-14T17:30:04.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:30:07.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-02-14T17:30:07.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:30:07.836 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-02-14T17:30:10.144 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-02-14T17:30:19.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-02-14T17:30:19.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:30:22.147 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-02-14T17:30:22.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:30:38.272 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-02-14T17:30:40.572 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-02-14T17:30:53.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-02-14T17:30:53.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:30:53.741 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-02-14T17:30:54.156 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-02-14T17:30:54.156 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-420.el9.x86_64"... 2024-02-14T17:30:54.156 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-02-14T17:30:54.170 INFO:teuthology.orchestra.run.smithi179.stdout:5.14.0-420.el9.x86_64 2024-02-14T17:30:54.171 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-420.el9.x86_64 vs 5.14.0-420.el9.x86_64 2024-02-14T17:30:54.171 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-14T17:30:54.171 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-14T17:30:55.172 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-14T17:30:55.172 DEBUG:teuthology.orchestra.run.smithi179:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-14T17:30:55.374 INFO:teuthology.orchestra.run.smithi179.stdout:ttyS1 2024-02-14T17:30:55.418 DEBUG:teuthology.parallel:result is None 2024-02-14T17:30:55.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-02-14T17:30:55.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:30:55.892 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-02-14T17:30:56.304 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-02-14T17:30:56.304 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-420.el9.x86_64"... 2024-02-14T17:30:56.304 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-02-14T17:30:56.318 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-420.el9.x86_64 2024-02-14T17:30:56.318 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-420.el9.x86_64 vs 5.14.0-420.el9.x86_64 2024-02-14T17:30:56.319 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-14T17:30:56.319 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-14T17:30:57.319 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-14T17:30:57.320 DEBUG:teuthology.orchestra.run.smithi100:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-14T17:30:57.423 INFO:teuthology.orchestra.run.smithi100.stdout:ttyS1 2024-02-14T17:30:57.461 DEBUG:teuthology.parallel:result is None 2024-02-14T17:30:57.461 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-14T17:30:57.468 INFO:teuthology.task.internal:Creating test directory... 2024-02-14T17:30:57.468 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-14T17:30:57.471 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-14T17:30:57.494 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-14T17:30:57.502 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-14T17:30:57.525 INFO:teuthology.task.internal:Creating archive directory... 2024-02-14T17:30:57.526 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-14T17:30:57.537 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-14T17:30:57.566 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-14T17:30:57.574 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-14T17:30:57.575 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-14T17:30:57.604 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-14T17:30:57.632 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:30:57.632 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:30:57.645 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:30:57.645 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-14T17:30:57.647 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-14T17:30:57.655 INFO:teuthology.task.internal:Configuring sudo... 2024-02-14T17:30:57.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-14T17:30:57.690 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-14T17:30:57.716 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-14T17:30:57.726 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-14T17:30:57.726 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-14T17:30:57.759 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-14T17:30:57.775 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:30:57.852 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:30:58.010 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:30:58.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:30:58.093 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:30:58.093 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-14T17:30:58.159 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:30:58.201 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-14T17:30:58.251 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:30:58.338 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-14T17:30:58.383 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:30:58.383 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-14T17:30:58.454 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-02-14T17:30:58.456 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-02-14T17:30:58.498 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T17:30:58.556 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T17:30:58.939 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-14T17:30:58.947 INFO:teuthology.task.internal:Starting timer... 2024-02-14T17:30:58.947 INFO:teuthology.run_tasks:Running task pcp... 2024-02-14T17:30:58.978 INFO:teuthology.run_tasks:Running task selinux... 2024-02-14T17:30:59.011 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-14T17:30:59.013 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-02-14T17:30:59.087 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-02-14T17:30:59.089 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2024-02-14T17:30:59.140 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2024-02-14T17:30:59.142 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-14T17:30:59.142 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-02-14T17:30:59.166 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-02-14T17:30:59.166 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2024-02-14T17:30:59.194 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2024-02-14T17:30:59.194 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2024-02-14T17:30:59.194 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T17:30:59.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:30:59.234 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T17:30:59.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:30:59.266 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-14T17:30:59.266 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-02-14T17:30:59.310 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2024-02-14T17:30:59.346 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-14T17:30:59.357 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-14T17:30:59.374 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-14T17:30:59.376 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-14T17:35:21.569 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-02-14T17:35:21.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-02-14T17:35:21.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:35:21.659 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-02-14T17:35:21.734 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-02-14T17:35:21.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-02-14T17:35:21.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T17:35:21.827 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-02-14T17:35:21.907 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-02-14T17:35:21.907 INFO:teuthology.run_tasks:Running task clock... 2024-02-14T17:35:21.919 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-14T17:35:21.919 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-14T17:35:21.919 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T17:35:21.922 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-14T17:35:21.922 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T17:35:21.951 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-14T17:35:21.967 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-14T17:35:21.980 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-14T17:35:21.996 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-02-14T17:35:21.997 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-14T17:35:22.009 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-02-14T17:35:22.025 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-14T17:35:22.026 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2024-02-14T17:35:22.040 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2024-02-14T17:35:22.041 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-14T17:35:22.057 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-14T17:35:22.073 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-14T17:35:22.101 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-02-14T17:35:22.104 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T17:35:22.104 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-02-14T17:35:22.104 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:35:22.104 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:35:22.105 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:35:22.105 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:35:22.124 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-02-14T17:35:22.127 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T17:35:22.128 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-02-14T17:35:22.128 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:35:22.128 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:35:22.128 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:35:22.128 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T17:35:22.128 INFO:teuthology.run_tasks:Running task pexec... 2024-02-14T17:35:22.143 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-14T17:35:22.144 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi100.front.sepia.ceph.com 2024-02-14T17:35:22.144 DEBUG:teuthology.orchestra.run.smithi100:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-14T17:35:22.145 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi179.front.sepia.ceph.com 2024-02-14T17:35:22.146 DEBUG:teuthology.orchestra.run.smithi179:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-14T17:35:22.743 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:00:51 ago on Wed 14 Feb 2024 05:34:31 PM UTC. 2024-02-14T17:35:22.764 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:51 ago on Wed 14 Feb 2024 05:34:31 PM UTC. 2024-02-14T17:35:22.905 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:35:22.906 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:35:22.906 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-02-14T17:35:22.906 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:35:22.906 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:35:22.907 INFO:teuthology.orchestra.run.smithi100.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-14T17:35:22.907 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:35:22.907 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:35:22.907 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:35:22.907 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-02-14T17:35:22.907 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:35:22.907 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 3.1 M 2024-02-14T17:35:22.907 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 9.8 M 2024-02-14T17:35:22.907 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:35:22.924 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:35:22.926 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:35:22.926 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-02-14T17:35:22.926 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:35:22.926 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:35:22.926 INFO:teuthology.orchestra.run.smithi179.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-14T17:35:22.926 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:35:22.926 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:35:22.926 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:35:22.926 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-02-14T17:35:22.926 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:35:22.927 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 3.1 M 2024-02-14T17:35:22.927 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 9.8 M 2024-02-14T17:35:22.927 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:35:23.364 INFO:teuthology.orchestra.run.smithi100.stdout:runc-1.1.12-1.el9.x86_64.rpm 6.8 MB/s | 3.1 MB 00:00 2024-02-14T17:35:23.365 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:35:23.365 INFO:teuthology.orchestra.run.smithi100.stdout:Total 6.8 MB/s | 3.1 MB 00:00 2024-02-14T17:35:23.365 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:35:23.367 INFO:teuthology.orchestra.run.smithi179.stdout:runc-1.1.12-1.el9.x86_64.rpm 7.1 MB/s | 3.1 MB 00:00 2024-02-14T17:35:23.368 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:35:23.368 INFO:teuthology.orchestra.run.smithi179.stdout:Total 7.1 MB/s | 3.1 MB 00:00 2024-02-14T17:35:23.368 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:35:23.371 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:35:23.371 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:35:23.373 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:35:23.374 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:35:23.442 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:35:23.442 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:35:23.444 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:35:23.444 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:35:23.640 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:35:23.644 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:35:23.737 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:35:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:35:24.282 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:35:24.301 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:35:24.591 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:35:24.591 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:35:24.591 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:35:24.591 INFO:teuthology.orchestra.run.smithi179.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-14T17:35:24.592 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:35:24.592 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:35:24.605 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-14T17:35:24.605 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:35:24.605 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:35:24.605 INFO:teuthology.orchestra.run.smithi100.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-14T17:35:24.605 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:35:24.605 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:35:24.839 DEBUG:teuthology.parallel:result is None 2024-02-14T17:35:24.879 DEBUG:teuthology.parallel:result is None 2024-02-14T17:35:24.879 INFO:teuthology.run_tasks:Running task install... 2024-02-14T17:35:24.888 DEBUG:teuthology.task.install:project ceph 2024-02-14T17:35:24.888 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1'}} 2024-02-14T17:35:24.888 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1'} 2024-02-14T17:35:24.889 INFO:teuthology.task.install:Using flavor: default 2024-02-14T17:35:24.894 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-14T17:35:24.894 INFO:teuthology.task.install:extra packages: [] 2024-02-14T17:35:24.894 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1', 'tag': None, 'wait_for_package': False} 2024-02-14T17:35:24.895 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:35:24.897 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1', 'tag': None, 'wait_for_package': False} 2024-02-14T17:35:24.897 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:35:25.044 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/ 2024-02-14T17:35:25.044 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1430.g8727ce93 2024-02-14T17:35:25.047 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/ 2024-02-14T17:35:25.047 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1430.g8727ce93 2024-02-14T17:35:25.154 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-14T17:35:25.155 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:35:25.155 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-14T17:35:25.182 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-14T17:35:25.182 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8727ce939fc044a98f215e29696bb1433f5758d1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-14T17:35:25.250 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-14T17:35:25.322 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-14T17:35:25.323 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:35:25.323 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-14T17:35:25.329 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-14T17:35:25.360 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-14T17:35:25.360 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8727ce939fc044a98f215e29696bb1433f5758d1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-14T17:35:25.397 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2024-02-14T17:35:25.398 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2024-02-14T17:35:25.429 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-14T17:35:25.510 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-14T17:35:25.577 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-02-14T17:35:25.579 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-02-14T17:35:25.654 INFO:teuthology.orchestra.run.smithi179.stdout:74 files removed 2024-02-14T17:35:25.681 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw 2024-02-14T17:35:25.819 INFO:teuthology.orchestra.run.smithi100.stdout:74 files removed 2024-02-14T17:35:25.845 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw 2024-02-14T17:35:26.456 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 180 kB/s | 82 kB 00:00 2024-02-14T17:35:26.729 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 144 kB/s | 82 kB 00:00 2024-02-14T17:35:26.790 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 35 kB/s | 10 kB 00:00 2024-02-14T17:35:27.254 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 21 kB/s | 10 kB 00:00 2024-02-14T17:35:27.265 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-02-14T17:35:27.696 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-02-14T17:35:28.390 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-02-14T17:35:29.176 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - AppStream 3.6 MB/s | 6.7 MB 00:01 2024-02-14T17:35:30.099 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2024-02-14T17:35:30.960 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-02-14T17:35:40.350 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 824 kB/s | 8.0 MB 00:09 2024-02-14T17:35:43.164 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-02-14T17:35:43.941 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 646 kB/s | 8.0 MB 00:12 2024-02-14T17:35:47.009 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 19 MB 00:01 2024-02-14T17:35:52.212 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.1 MB 00:01 2024-02-14T17:35:54.060 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 15 kB 00:00 2024-02-14T17:35:54.506 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 797 kB/s | 6.1 MB 00:07 2024-02-14T17:35:55.747 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-02-14T17:35:56.148 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-02-14T17:35:57.258 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2024-02-14T17:36:00.695 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-02-14T17:36:02.200 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-02-14T17:36:03.320 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:36:03.323 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:36:03.323 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:36:03.323 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:36:03.323 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:36:03.323 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 9.7 M 2024-02-14T17:36:03.323 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-02-14T17:36:03.323 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.4 M 2024-02-14T17:36:03.323 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.1 M 2024-02-14T17:36:03.323 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-02-14T17:36:03.323 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 5.3 M 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 20 M 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 25 k 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 743 k 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-14T17:36:03.324 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 499 k 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 5.1 M 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 45 k 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 131 k 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 162 k 2024-02-14T17:36:03.325 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 320 k 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 299 k 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 99 k 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout:Installing weak dependencies: 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:36:03.326 INFO:teuthology.orchestra.run.smithi179.stdout:Install 30 Packages 2024-02-14T17:36:03.327 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 2 Packages 2024-02-14T17:36:03.327 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:03.327 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 58 M 2024-02-14T17:36:03.327 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:36:04.288 INFO:teuthology.orchestra.run.smithi179.stdout:(1/32): ceph-base-19.0.0-1430.g8727ce93.el9.x86 5.8 MB/s | 5.3 MB 00:00 2024-02-14T17:36:04.338 INFO:teuthology.orchestra.run.smithi179.stdout:(2/32): ceph-selinux-19.0.0-1430.g8727ce93.el9. 505 kB/s | 25 kB 00:00 2024-02-14T17:36:04.463 INFO:teuthology.orchestra.run.smithi179.stdout:(3/32): libcephfs2-19.0.0-1430.g8727ce93.el9.x8 5.8 MB/s | 743 kB 00:00 2024-02-14T17:36:04.572 INFO:teuthology.orchestra.run.smithi179.stdout:(4/32): libradosstriper1-19.0.0-1430.g8727ce93. 4.5 MB/s | 499 kB 00:00 2024-02-14T17:36:04.705 INFO:teuthology.orchestra.run.smithi179.stdout:(5/32): ceph-radosgw-19.0.0-1430.g8727ce93.el9. 7.3 MB/s | 9.7 MB 00:01 2024-02-14T17:36:04.781 INFO:teuthology.orchestra.run.smithi179.stdout:(6/32): python3-ceph-argparse-19.0.0-1430.g8727 605 kB/s | 45 kB 00:00 2024-02-14T17:36:04.817 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:36:04.819 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:36:04.819 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:36:04.820 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:36:04.820 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:36:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 9.7 M 2024-02-14T17:36:04.820 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-02-14T17:36:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.4 M 2024-02-14T17:36:04.820 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.1 M 2024-02-14T17:36:04.821 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-14T17:36:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options 2024-02-14T17:36:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-14T17:36:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 5.3 M 2024-02-14T17:36:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 20 M 2024-02-14T17:36:04.821 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 25 k 2024-02-14T17:36:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-14T17:36:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-14T17:36:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-14T17:36:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 743 k 2024-02-14T17:36:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-14T17:36:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-14T17:36:04.823 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-14T17:36:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 499 k 2024-02-14T17:36:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-14T17:36:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 5.1 M 2024-02-14T17:36:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-14T17:36:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-14T17:36:04.824 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-14T17:36:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse 2024-02-14T17:36:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 45 k 2024-02-14T17:36:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 131 k 2024-02-14T17:36:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 162 k 2024-02-14T17:36:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-14T17:36:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 320 k 2024-02-14T17:36:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 299 k 2024-02-14T17:36:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 99 k 2024-02-14T17:36:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-14T17:36:04.825 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-14T17:36:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-14T17:36:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-14T17:36:04.826 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-02-14T17:36:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-14T17:36:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:36:04.826 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:36:04.826 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:36:04.826 INFO:teuthology.orchestra.run.smithi100.stdout:Install 30 Packages 2024-02-14T17:36:04.826 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 2 Packages 2024-02-14T17:36:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:36:04.827 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 58 M 2024-02-14T17:36:04.827 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:36:04.839 INFO:teuthology.orchestra.run.smithi179.stdout:(7/32): python3-ceph-common-19.0.0-1430.g8727ce 2.2 MB/s | 131 kB 00:00 2024-02-14T17:36:04.914 INFO:teuthology.orchestra.run.smithi179.stdout:(8/32): python3-cephfs-19.0.0-1430.g8727ce93.el 2.1 MB/s | 162 kB 00:00 2024-02-14T17:36:05.006 INFO:teuthology.orchestra.run.smithi179.stdout:(9/32): python3-rados-19.0.0-1430.g8727ce93.el9 3.4 MB/s | 320 kB 00:00 2024-02-14T17:36:05.098 INFO:teuthology.orchestra.run.smithi179.stdout:(10/32): python3-rbd-19.0.0-1430.g8727ce93.el9. 3.2 MB/s | 299 kB 00:00 2024-02-14T17:36:05.182 INFO:teuthology.orchestra.run.smithi179.stdout:(11/32): python3-rgw-19.0.0-1430.g8727ce93.el9. 1.2 MB/s | 99 kB 00:00 2024-02-14T17:36:05.257 INFO:teuthology.orchestra.run.smithi179.stdout:(12/32): librgw2-19.0.0-1430.g8727ce93.el9.x86_ 7.4 MB/s | 5.1 MB 00:00 2024-02-14T17:36:05.374 INFO:teuthology.orchestra.run.smithi179.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 551 kB/s | 106 kB 00:00 2024-02-14T17:36:05.408 INFO:teuthology.orchestra.run.smithi179.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-02-14T17:36:05.433 INFO:teuthology.orchestra.run.smithi179.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 759 kB/s | 44 kB 00:00 2024-02-14T17:36:05.483 INFO:teuthology.orchestra.run.smithi179.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-02-14T17:36:05.509 INFO:teuthology.orchestra.run.smithi179.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-02-14T17:36:05.534 INFO:teuthology.orchestra.run.smithi179.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 827 kB/s | 41 kB 00:00 2024-02-14T17:36:05.559 INFO:teuthology.orchestra.run.smithi179.stdout:(19/32): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-02-14T17:36:05.584 INFO:teuthology.orchestra.run.smithi179.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 678 kB/s | 34 kB 00:00 2024-02-14T17:36:05.610 INFO:teuthology.orchestra.run.smithi179.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 868 kB/s | 43 kB 00:00 2024-02-14T17:36:06.027 INFO:teuthology.orchestra.run.smithi179.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 699 kB/s | 309 kB 00:00 2024-02-14T17:36:06.102 INFO:teuthology.orchestra.run.smithi179.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-02-14T17:36:06.177 INFO:teuthology.orchestra.run.smithi179.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 648 kB/s | 49 kB 00:00 2024-02-14T17:36:06.261 INFO:teuthology.orchestra.run.smithi179.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 807 kB/s | 67 kB 00:00 2024-02-14T17:36:06.336 INFO:teuthology.orchestra.run.smithi179.stdout:(26/32): libarrow-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-02-14T17:36:06.412 INFO:teuthology.orchestra.run.smithi179.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-02-14T17:36:06.454 INFO:teuthology.orchestra.run.smithi179.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-02-14T17:36:06.521 INFO:teuthology.orchestra.run.smithi179.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-02-14T17:36:07.346 INFO:teuthology.orchestra.run.smithi179.stdout:(30/32): librados2-19.0.0-1430.g8727ce93.el9.x8 3.8 MB/s | 3.4 MB 00:00 2024-02-14T17:36:07.904 INFO:teuthology.orchestra.run.smithi179.stdout:(31/32): ceph-common-19.0.0-1430.g8727ce93.el9. 4.4 MB/s | 20 MB 00:04 2024-02-14T17:36:07.954 INFO:teuthology.orchestra.run.smithi179.stdout:(32/32): librbd1-19.0.0-1430.g8727ce93.el9.x86_ 2.2 MB/s | 3.1 MB 00:01 2024-02-14T17:36:07.961 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:36:07.961 INFO:teuthology.orchestra.run.smithi179.stdout:Total 12 MB/s | 58 MB 00:04 2024-02-14T17:36:07.961 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:36:07.998 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:36:07.998 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:36:08.478 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:36:08.479 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:36:08.758 INFO:teuthology.orchestra.run.smithi100.stdout:(1/32): ceph-base-19.0.0-1430.g8727ce93.el9.x86 1.4 MB/s | 5.3 MB 00:03 2024-02-14T17:36:08.850 INFO:teuthology.orchestra.run.smithi100.stdout:(2/32): ceph-selinux-19.0.0-1430.g8727ce93.el9. 275 kB/s | 25 kB 00:00 2024-02-14T17:36:09.084 INFO:teuthology.orchestra.run.smithi100.stdout:(3/32): libcephfs2-19.0.0-1430.g8727ce93.el9.x8 3.1 MB/s | 743 kB 00:00 2024-02-14T17:36:09.259 INFO:teuthology.orchestra.run.smithi100.stdout:(4/32): libradosstriper1-19.0.0-1430.g8727ce93. 2.8 MB/s | 499 kB 00:00 2024-02-14T17:36:09.659 INFO:teuthology.orchestra.run.smithi100.stdout:(5/32): ceph-radosgw-19.0.0-1430.g8727ce93.el9. 2.1 MB/s | 9.7 MB 00:04 2024-02-14T17:36:09.751 INFO:teuthology.orchestra.run.smithi100.stdout:(6/32): python3-ceph-argparse-19.0.0-1430.g8727 494 kB/s | 45 kB 00:00 2024-02-14T17:36:09.890 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:36:09.910 INFO:teuthology.orchestra.run.smithi100.stdout:(7/32): python3-ceph-common-19.0.0-1430.g8727ce 830 kB/s | 131 kB 00:00 2024-02-14T17:36:09.963 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-02-14T17:36:09.992 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-02-14T17:36:10.034 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-02-14T17:36:10.057 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-14T17:36:10.069 INFO:teuthology.orchestra.run.smithi100.stdout:(8/32): python3-cephfs-19.0.0-1430.g8727ce93.el 1.0 MB/s | 162 kB 00:00 2024-02-14T17:36:10.072 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-14T17:36:10.144 INFO:teuthology.orchestra.run.smithi100.stdout:(9/32): librgw2-19.0.0-1430.g8727ce93.el9.x86_6 5.8 MB/s | 5.1 MB 00:00 2024-02-14T17:36:10.244 INFO:teuthology.orchestra.run.smithi100.stdout:(10/32): python3-rbd-19.0.0-1430.g8727ce93.el9. 2.9 MB/s | 299 kB 00:00 2024-02-14T17:36:10.268 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-14T17:36:10.295 INFO:teuthology.orchestra.run.smithi100.stdout:(11/32): python3-rados-19.0.0-1430.g8727ce93.el 1.4 MB/s | 320 kB 00:00 2024-02-14T17:36:10.320 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-14T17:36:10.320 INFO:teuthology.orchestra.run.smithi100.stdout:(12/32): python3-rgw-19.0.0-1430.g8727ce93.el9. 1.3 MB/s | 99 kB 00:00 2024-02-14T17:36:10.445 INFO:teuthology.orchestra.run.smithi100.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 705 kB/s | 106 kB 00:00 2024-02-14T17:36:10.470 INFO:teuthology.orchestra.run.smithi100.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-02-14T17:36:10.487 INFO:teuthology.orchestra.run.smithi100.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-14T17:36:10.554 INFO:teuthology.orchestra.run.smithi100.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-02-14T17:36:10.588 INFO:teuthology.orchestra.run.smithi100.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 282 kB 00:00 2024-02-14T17:36:10.605 INFO:teuthology.orchestra.run.smithi100.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 825 kB/s | 41 kB 00:00 2024-02-14T17:36:10.630 INFO:teuthology.orchestra.run.smithi100.stdout:(19/32): qatlib-23.11.0-1.el9.x86_64.rpm 5.1 MB/s | 215 kB 00:00 2024-02-14T17:36:10.647 INFO:teuthology.orchestra.run.smithi100.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 814 kB/s | 34 kB 00:00 2024-02-14T17:36:10.664 INFO:teuthology.orchestra.run.smithi100.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-02-14T17:36:10.700 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-02-14T17:36:10.728 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:36:10.820 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:36:10.909 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 8/34 2024-02-14T17:36:10.938 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 9/34 2024-02-14T17:36:10.948 INFO:teuthology.orchestra.run.smithi100.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.0 MB/s | 309 kB 00:00 2024-02-14T17:36:11.000 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 9/34 2024-02-14T17:36:11.006 INFO:teuthology.orchestra.run.smithi100.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-02-14T17:36:11.029 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-02-14T17:36:11.065 INFO:teuthology.orchestra.run.smithi100.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 834 kB/s | 49 kB 00:00 2024-02-14T17:36:11.073 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9. 11/34 2024-02-14T17:36:11.142 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 12/34 2024-02-14T17:36:11.165 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 13/34 2024-02-14T17:36:11.227 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 13/34 2024-02-14T17:36:11.255 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-02-14T17:36:11.257 INFO:teuthology.orchestra.run.smithi100.stdout:(25/32): ceph-common-19.0.0-1430.g8727ce93.el9. 3.2 MB/s | 20 MB 00:06 2024-02-14T17:36:11.282 INFO:teuthology.orchestra.run.smithi100.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 310 kB/s | 67 kB 00:00 2024-02-14T17:36:11.349 INFO:teuthology.orchestra.run.smithi100.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-02-14T17:36:11.365 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-02-14T17:36:11.405 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-02-14T17:36:11.456 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-02-14T17:36:11.487 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-02-14T17:36:11.516 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-02-14T17:36:11.549 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-02-14T17:36:11.558 INFO:teuthology.orchestra.run.smithi100.stdout:(28/32): libarrow-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 4.4 MB 00:00 2024-02-14T17:36:11.750 INFO:teuthology.orchestra.run.smithi100.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-02-14T17:36:11.866 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-02-14T17:36:11.875 INFO:teuthology.orchestra.run.smithi100.stdout:(30/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.3 MB/s | 838 kB 00:00 2024-02-14T17:36:11.895 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 22/34 2024-02-14T17:36:11.950 INFO:teuthology.orchestra.run.smithi100.stdout:(31/32): librados2-19.0.0-1430.g8727ce93.el9.x8 8.6 MB/s | 3.4 MB 00:00 2024-02-14T17:36:11.986 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 22/34 2024-02-14T17:36:12.017 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 23/34 2024-02-14T17:36:12.055 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-02-14T17:36:12.183 INFO:teuthology.orchestra.run.smithi100.stdout:(32/32): librbd1-19.0.0-1430.g8727ce93.el9.x86_ 7.2 MB/s | 3.1 MB 00:00 2024-02-14T17:36:12.191 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:36:12.191 INFO:teuthology.orchestra.run.smithi100.stdout:Total 7.8 MB/s | 58 MB 00:07 2024-02-14T17:36:12.191 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:36:12.228 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:36:12.228 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:36:12.508 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-02-14T17:36:12.531 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 26/34 2024-02-14T17:36:12.594 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 26/34 2024-02-14T17:36:12.662 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 27/34 2024-02-14T17:36:12.692 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x8 28/34 2024-02-14T17:36:12.704 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:36:12.704 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:36:14.114 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:36:14.209 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-02-14T17:36:14.247 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-02-14T17:36:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-02-14T17:36:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-14T17:36:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-02-14T17:36:14.464 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-14T17:36:14.518 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-02-14T17:36:14.851 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:36:14.897 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:36:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-02-14T17:36:14.949 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:36:15.023 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:36:15.111 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 8/34 2024-02-14T17:36:15.142 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 9/34 2024-02-14T17:36:15.203 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 9/34 2024-02-14T17:36:15.233 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-02-14T17:36:15.277 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9. 11/34 2024-02-14T17:36:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 12/34 2024-02-14T17:36:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 13/34 2024-02-14T17:36:15.440 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 13/34 2024-02-14T17:36:15.463 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:36:15.476 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-02-14T17:36:15.508 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 30/34 2024-02-14T17:36:15.592 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-02-14T17:36:15.599 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 30/34 2024-02-14T17:36:15.599 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-14T17:36:15.600 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-14T17:36:15.600 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:15.626 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:36:15.635 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-02-14T17:36:15.677 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-02-14T17:36:15.709 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-02-14T17:36:15.738 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-02-14T17:36:15.771 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-02-14T17:36:16.095 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-02-14T17:36:16.117 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 22/34 2024-02-14T17:36:16.208 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 22/34 2024-02-14T17:36:16.239 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 23/34 2024-02-14T17:36:16.294 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-02-14T17:36:16.774 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-02-14T17:36:16.804 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 26/34 2024-02-14T17:36:16.875 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 26/34 2024-02-14T17:36:16.951 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 27/34 2024-02-14T17:36:16.981 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x8 28/34 2024-02-14T17:36:19.071 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:36:19.127 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:36:19.711 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 29/34 2024-02-14T17:36:19.772 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 30/34 2024-02-14T17:36:19.854 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 30/34 2024-02-14T17:36:19.854 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-14T17:36:19.855 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-14T17:36:19.855 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:36:19.890 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:36:25.363 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:36:25.363 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2024-02-14T17:36:25.363 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2024-02-14T17:36:25.364 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2024-02-14T17:36:25.364 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2024-02-14T17:36:25.364 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2024-02-14T17:36:25.364 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /root 2024-02-14T17:36:25.364 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2024-02-14T17:36:25.364 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:26.310 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 32/34 2024-02-14T17:36:26.334 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 32/34 2024-02-14T17:36:26.334 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:36:26.334 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-14T17:36:26.334 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T17:36:26.334 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T17:36:26.334 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:26.336 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-14T17:36:26.437 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-14T17:36:26.437 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:36:27.631 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:36:27.632 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 1/34 2024-02-14T17:36:27.632 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2/34 2024-02-14T17:36:27.632 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 3/34 2024-02-14T17:36:27.636 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 4/34 2024-02-14T17:36:27.636 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 5/34 2024-02-14T17:36:27.636 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 6/34 2024-02-14T17:36:27.636 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:36:27.636 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9. 8/34 2024-02-14T17:36:27.636 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x8 9/34 2024-02-14T17:36:27.636 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 10/34 2024-02-14T17:36:27.636 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 11/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 12/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 13/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-02-14T17:36:27.637 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-02-14T17:36:27.638 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-02-14T17:36:27.638 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-02-14T17:36:27.638 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-02-14T17:36:27.638 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-02-14T17:36:27.638 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-02-14T17:36:27.638 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:36:27.639 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-02-14T17:36:27.639 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 33/34 2024-02-14T17:36:28.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:36:28.563 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.564 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.565 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.566 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:28.566 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-14T17:36:28.566 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-14T17:36:28.566 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-14T17:36:28.566 INFO:teuthology.orchestra.run.smithi179.stdout: re2-1:20211101-3.el9.x86_64 2024-02-14T17:36:28.566 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-14T17:36:28.566 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:28.566 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:36:28.830 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-test 2024-02-14T17:36:29.403 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:00:29 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:36:29.572 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:36:29.572 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 47 M 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 163 k 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout:Install 6 Packages 2024-02-14T17:36:29.573 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:29.574 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 48 M 2024-02-14T17:36:29.574 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 204 M 2024-02-14T17:36:29.574 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:36:29.646 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:36:29.646 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-02-14T17:36:29.646 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-02-14T17:36:29.646 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-02-14T17:36:29.646 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-02-14T17:36:29.646 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-02-14T17:36:29.646 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-02-14T17:36:29.646 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-02-14T17:36:29.647 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:36:29.841 INFO:teuthology.orchestra.run.smithi179.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2024-02-14T17:36:29.908 INFO:teuthology.orchestra.run.smithi179.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 937 kB/s | 63 kB 00:00 2024-02-14T17:36:30.008 INFO:teuthology.orchestra.run.smithi179.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 1.8 MB/s | 186 kB 00:00 2024-02-14T17:36:30.100 INFO:teuthology.orchestra.run.smithi179.stdout:(4/6): libcephsqlite-19.0.0-1430.g8727ce93.el9. 310 kB/s | 163 kB 00:00 2024-02-14T17:36:30.125 INFO:teuthology.orchestra.run.smithi179.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.8 MB/s | 217 kB 00:00 2024-02-14T17:36:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 32/34 2024-02-14T17:36:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 32/34 2024-02-14T17:36:30.585 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:36:30.585 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-14T17:36:30.585 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T17:36:30.585 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-14T17:36:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:36:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-14T17:36:30.680 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-02-14T17:36:30.680 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:36:31.821 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:36:31.821 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 1/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 3/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 4/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 5/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_6 6/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 7/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9. 8/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x8 9/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 10/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 11/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 12/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 13/34 2024-02-14T17:36:31.822 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-02-14T17:36:31.823 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-02-14T17:36:31.824 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-02-14T17:36:31.824 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 31/34 2024-02-14T17:36:31.824 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-02-14T17:36:31.824 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 33/34 2024-02-14T17:36:32.894 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-02-14T17:36:32.894 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:36:32.894 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-02-14T17:36:32.894 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.894 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-14T17:36:32.895 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.896 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:32.897 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-14T17:36:32.897 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-14T17:36:32.897 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-14T17:36:32.897 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2024-02-14T17:36:32.897 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-14T17:36:32.897 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:36:32.897 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:36:33.295 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-test 2024-02-14T17:36:33.862 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:00:31 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:36:34.034 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:36:34.035 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:36:34.035 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:36:34.035 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:36:34.035 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:36:34.035 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 47 M 2024-02-14T17:36:34.035 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-14T17:36:34.035 INFO:teuthology.orchestra.run.smithi100.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-14T17:36:34.035 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 163 k 2024-02-14T17:36:34.035 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-14T17:36:34.036 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-14T17:36:34.036 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-14T17:36:34.036 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:36:34.036 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:36:34.036 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:36:34.036 INFO:teuthology.orchestra.run.smithi100.stdout:Install 6 Packages 2024-02-14T17:36:34.036 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:36:34.037 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 48 M 2024-02-14T17:36:34.037 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 204 M 2024-02-14T17:36:34.037 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:36:34.226 INFO:teuthology.orchestra.run.smithi100.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 300 kB 00:00 2024-02-14T17:36:34.276 INFO:teuthology.orchestra.run.smithi100.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-02-14T17:36:34.343 INFO:teuthology.orchestra.run.smithi100.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 2.7 MB/s | 186 kB 00:00 2024-02-14T17:36:34.419 INFO:teuthology.orchestra.run.smithi100.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-02-14T17:36:34.627 INFO:teuthology.orchestra.run.smithi100.stdout:(5/6): libcephsqlite-19.0.0-1430.g8727ce93.el9. 276 kB/s | 163 kB 00:00 2024-02-14T17:36:51.033 INFO:teuthology.orchestra.run.smithi179.stdout:(6/6): ceph-test-19.0.0-1430.g8727ce93.el9.x86_ 2.2 MB/s | 47 MB 00:21 2024-02-14T17:36:51.034 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:36:51.035 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.2 MB/s | 48 MB 00:21 2024-02-14T17:36:51.035 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:36:51.052 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:36:51.053 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:36:51.446 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:36:51.447 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:36:52.030 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:36:52.093 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-14T17:36:52.130 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-14T17:36:52.183 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-14T17:36:52.228 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-14T17:36:52.252 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 5/6 2024-02-14T17:36:52.509 INFO:teuthology.orchestra.run.smithi100.stdout:(6/6): ceph-test-19.0.0-1430.g8727ce93.el9.x86_ 2.6 MB/s | 47 MB 00:18 2024-02-14T17:36:52.511 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:36:52.512 INFO:teuthology.orchestra.run.smithi100.stdout:Total 2.6 MB/s | 48 MB 00:18 2024-02-14T17:36:52.512 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:36:52.532 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:36:52.533 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:36:52.928 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:36:52.928 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:36:53.501 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:36:53.574 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-14T17:36:53.608 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-14T17:36:53.654 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-14T17:36:53.698 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-14T17:36:53.722 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 5/6 2024-02-14T17:36:56.674 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 5/6 2024-02-14T17:36:56.759 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 6/6 2024-02-14T17:36:58.178 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 5/6 2024-02-14T17:36:58.255 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 6/6 2024-02-14T17:36:59.209 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 6/6 2024-02-14T17:36:59.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 1/6 2024-02-14T17:36:59.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 2/6 2024-02-14T17:36:59.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-14T17:36:59.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-14T17:36:59.209 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-14T17:36:59.622 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-14T17:36:59.622 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:59.622 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:36:59.622 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:59.622 INFO:teuthology.orchestra.run.smithi179.stdout: jq-1.6-16.el9.x86_64 2024-02-14T17:36:59.622 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:36:59.623 INFO:teuthology.orchestra.run.smithi179.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-14T17:36:59.623 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-14T17:36:59.623 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-14T17:36:59.623 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:36:59.623 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:36:59.851 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph 2024-02-14T17:37:00.437 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:00 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:00.530 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 6/6 2024-02-14T17:37:00.530 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 1/6 2024-02-14T17:37:00.530 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 2/6 2024-02-14T17:37:00.530 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-14T17:37:00.531 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-14T17:37:00.531 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-14T17:37:00.613 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:00.615 INFO:teuthology.orchestra.run.smithi179.stdout:========================================================================================= 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout:========================================================================================= 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 6.6 k 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 2.1 M 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 1.5 M 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 247 k 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 4.7 M 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 17 M 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-14T17:37:00.616 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-14T17:37:00.617 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-14T17:37:00.618 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:00.619 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:00.619 INFO:teuthology.orchestra.run.smithi179.stdout:========================================================================================= 2024-02-14T17:37:00.619 INFO:teuthology.orchestra.run.smithi179.stdout:Install 36 Packages 2024-02-14T17:37:00.619 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:00.619 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 30 M 2024-02-14T17:37:00.619 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 107 M 2024-02-14T17:37:00.619 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:00.993 INFO:teuthology.orchestra.run.smithi179.stdout:(1/36): ceph-19.0.0-1430.g8727ce93.el9.x86_64.r 37 kB/s | 6.6 kB 00:00 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi100.stdout: jq-1.6-16.el9.x86_64 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:01.038 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:01.039 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:01.254 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph 2024-02-14T17:37:01.393 INFO:teuthology.orchestra.run.smithi179.stdout:(2/36): ceph-mgr-19.0.0-1430.g8727ce93.el9.x86_ 2.6 MB/s | 1.5 MB 00:00 2024-02-14T17:37:01.494 INFO:teuthology.orchestra.run.smithi179.stdout:(3/36): ceph-mds-19.0.0-1430.g8727ce93.el9.x86_ 3.1 MB/s | 2.1 MB 00:00 2024-02-14T17:37:01.561 INFO:teuthology.orchestra.run.smithi179.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1430.g8727 3.6 MB/s | 247 kB 00:00 2024-02-14T17:37:01.678 INFO:teuthology.orchestra.run.smithi179.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2024-02-14T17:37:01.719 INFO:teuthology.orchestra.run.smithi179.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-02-14T17:37:01.753 INFO:teuthology.orchestra.run.smithi179.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 944 kB/s | 32 kB 00:00 2024-02-14T17:37:01.787 INFO:teuthology.orchestra.run.smithi179.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-02-14T17:37:01.820 INFO:teuthology.orchestra.run.smithi179.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-02-14T17:37:01.837 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:00:59 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:01.862 INFO:teuthology.orchestra.run.smithi179.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-02-14T17:37:01.929 INFO:teuthology.orchestra.run.smithi179.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-02-14T17:37:01.996 INFO:teuthology.orchestra.run.smithi179.stdout:(12/36): ceph-mon-19.0.0-1430.g8727ce93.el9.x86 4.7 MB/s | 4.7 MB 00:01 2024-02-14T17:37:02.005 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:02.008 INFO:teuthology.orchestra.run.smithi100.stdout:========================================================================================= 2024-02-14T17:37:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:37:02.008 INFO:teuthology.orchestra.run.smithi100.stdout:========================================================================================= 2024-02-14T17:37:02.008 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:37:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 6.6 k 2024-02-14T17:37:02.008 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-14T17:37:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 2.1 M 2024-02-14T17:37:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 1.5 M 2024-02-14T17:37:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 247 k 2024-02-14T17:37:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 4.7 M 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 17 M 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-14T17:37:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-14T17:37:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-14T17:37:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-14T17:37:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-14T17:37:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:02.011 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:37:02.011 INFO:teuthology.orchestra.run.smithi100.stdout:========================================================================================= 2024-02-14T17:37:02.011 INFO:teuthology.orchestra.run.smithi100.stdout:Install 36 Packages 2024-02-14T17:37:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:02.011 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 30 M 2024-02-14T17:37:02.011 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 107 M 2024-02-14T17:37:02.012 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:37:02.022 INFO:teuthology.orchestra.run.smithi179.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-02-14T17:37:02.047 INFO:teuthology.orchestra.run.smithi179.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-02-14T17:37:02.073 INFO:teuthology.orchestra.run.smithi179.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2024-02-14T17:37:02.097 INFO:teuthology.orchestra.run.smithi179.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.6 MB/s | 187 kB 00:00 2024-02-14T17:37:02.256 INFO:teuthology.orchestra.run.smithi179.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 954 kB/s | 151 kB 00:00 2024-02-14T17:37:02.340 INFO:teuthology.orchestra.run.smithi179.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 521 kB/s | 43 kB 00:00 2024-02-14T17:37:02.398 INFO:teuthology.orchestra.run.smithi179.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-02-14T17:37:02.456 INFO:teuthology.orchestra.run.smithi100.stdout:(1/36): ceph-19.0.0-1430.g8727ce93.el9.x86_64.r 25 kB/s | 6.6 kB 00:00 2024-02-14T17:37:02.465 INFO:teuthology.orchestra.run.smithi179.stdout:(20/36): python3-cherrypy-18.6.1-2.el9.noarch.r 5.2 MB/s | 358 kB 00:00 2024-02-14T17:37:02.507 INFO:teuthology.orchestra.run.smithi179.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 256 kB/s | 11 kB 00:00 2024-02-14T17:37:02.549 INFO:teuthology.orchestra.run.smithi179.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 424 kB/s | 18 kB 00:00 2024-02-14T17:37:02.600 INFO:teuthology.orchestra.run.smithi179.stdout:(23/36): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-02-14T17:37:02.792 INFO:teuthology.orchestra.run.smithi179.stdout:(24/36): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-02-14T17:37:02.875 INFO:teuthology.orchestra.run.smithi179.stdout:(25/36): python3-jaraco-text-3.2.0-6.el9.noarch 237 kB/s | 20 kB 00:00 2024-02-14T17:37:02.917 INFO:teuthology.orchestra.run.smithi179.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 61 kB/s | 19 kB 00:00 2024-02-14T17:37:02.951 INFO:teuthology.orchestra.run.smithi179.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-02-14T17:37:02.976 INFO:teuthology.orchestra.run.smithi179.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-02-14T17:37:03.006 INFO:teuthology.orchestra.run.smithi100.stdout:(2/36): ceph-mon-19.0.0-1430.g8727ce93.el9.x86_ 8.5 MB/s | 4.7 MB 00:00 2024-02-14T17:37:03.018 INFO:teuthology.orchestra.run.smithi179.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 397 kB/s | 16 kB 00:00 2024-02-14T17:37:03.051 INFO:teuthology.orchestra.run.smithi179.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-02-14T17:37:03.077 INFO:teuthology.orchestra.run.smithi179.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-02-14T17:37:03.102 INFO:teuthology.orchestra.run.smithi179.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-02-14T17:37:03.127 INFO:teuthology.orchestra.run.smithi179.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-02-14T17:37:03.169 INFO:teuthology.orchestra.run.smithi179.stdout:(34/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 6.2 MB/s | 427 kB 00:00 2024-02-14T17:37:03.194 INFO:teuthology.orchestra.run.smithi179.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 299 kB/s | 20 kB 00:00 2024-02-14T17:37:03.331 INFO:teuthology.orchestra.run.smithi100.stdout:(3/36): ceph-mgr-19.0.0-1430.g8727ce93.el9.x86_ 1.3 MB/s | 1.5 MB 00:01 2024-02-14T17:37:03.428 INFO:teuthology.orchestra.run.smithi179.stdout:(36/36): ceph-osd-19.0.0-1430.g8727ce93.el9.x86 8.2 MB/s | 17 MB 00:02 2024-02-14T17:37:03.432 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:03.432 INFO:teuthology.orchestra.run.smithi179.stdout:Total 11 MB/s | 30 MB 00:02 2024-02-14T17:37:03.453 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:03.493 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:03.493 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:03.523 INFO:teuthology.orchestra.run.smithi100.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1430.g8727 1.3 MB/s | 247 kB 00:00 2024-02-14T17:37:03.565 INFO:teuthology.orchestra.run.smithi100.stdout:(5/36): ceph-mds-19.0.0-1430.g8727ce93.el9.x86_ 1.5 MB/s | 2.1 MB 00:01 2024-02-14T17:37:03.690 INFO:teuthology.orchestra.run.smithi100.stdout:(6/36): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-02-14T17:37:03.724 INFO:teuthology.orchestra.run.smithi100.stdout:(7/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 144 kB 00:00 2024-02-14T17:37:03.749 INFO:teuthology.orchestra.run.smithi100.stdout:(8/36): python3-markupsafe-1.1.1-12.el9.x86_64. 541 kB/s | 32 kB 00:00 2024-02-14T17:37:03.774 INFO:teuthology.orchestra.run.smithi100.stdout:(9/36): python3-pytz-2021.1-5.el9.noarch.rpm 943 kB/s | 47 kB 00:00 2024-02-14T17:37:03.799 INFO:teuthology.orchestra.run.smithi100.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 769 kB/s | 39 kB 00:00 2024-02-14T17:37:03.833 INFO:teuthology.orchestra.run.smithi100.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-02-14T17:37:03.892 INFO:teuthology.orchestra.run.smithi100.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2024-02-14T17:37:03.896 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:03.896 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:03.917 INFO:teuthology.orchestra.run.smithi100.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-02-14T17:37:03.942 INFO:teuthology.orchestra.run.smithi100.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-02-14T17:37:03.976 INFO:teuthology.orchestra.run.smithi100.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 1.9 MB/s | 113 kB 00:00 2024-02-14T17:37:04.001 INFO:teuthology.orchestra.run.smithi100.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 187 kB 00:00 2024-02-14T17:37:04.160 INFO:teuthology.orchestra.run.smithi100.stdout:(17/36): ceph-osd-19.0.0-1430.g8727ce93.el9.x86 15 MB/s | 17 MB 00:01 2024-02-14T17:37:04.461 INFO:teuthology.orchestra.run.smithi100.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 145 kB/s | 43 kB 00:00 2024-02-14T17:37:04.485 INFO:teuthology.orchestra.run.smithi100.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 312 kB/s | 151 kB 00:00 2024-02-14T17:37:04.661 INFO:teuthology.orchestra.run.smithi100.stdout:(20/36): python3-cherrypy-18.6.1-2.el9.noarch.r 2.0 MB/s | 358 kB 00:00 2024-02-14T17:37:04.686 INFO:teuthology.orchestra.run.smithi100.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 765 kB/s | 172 kB 00:00 2024-02-14T17:37:04.728 INFO:teuthology.orchestra.run.smithi100.stdout:(22/36): lua-devel-5.4.4-4.el9.x86_64.rpm 30 kB/s | 22 kB 00:00 2024-02-14T17:37:04.762 INFO:teuthology.orchestra.run.smithi100.stdout:(23/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 106 kB/s | 11 kB 00:00 2024-02-14T17:37:04.779 INFO:teuthology.orchestra.run.smithi100.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 191 kB/s | 18 kB 00:00 2024-02-14T17:37:04.854 INFO:teuthology.orchestra.run.smithi100.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 212 kB/s | 19 kB 00:00 2024-02-14T17:37:04.871 INFO:teuthology.orchestra.run.smithi100.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-02-14T17:37:04.946 INFO:teuthology.orchestra.run.smithi100.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 504 kB/s | 46 kB 00:00 2024-02-14T17:37:04.963 INFO:teuthology.orchestra.run.smithi100.stdout:(28/36): python3-jaraco-collections-3.0.0-8.el9 99 kB/s | 23 kB 00:00 2024-02-14T17:37:04.988 INFO:teuthology.orchestra.run.smithi100.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 672 kB/s | 79 kB 00:00 2024-02-14T17:37:05.047 INFO:teuthology.orchestra.run.smithi100.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-02-14T17:37:05.064 INFO:teuthology.orchestra.run.smithi100.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 163 kB/s | 16 kB 00:00 2024-02-14T17:37:05.089 INFO:teuthology.orchestra.run.smithi100.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 896 kB/s | 90 kB 00:00 2024-02-14T17:37:05.140 INFO:teuthology.orchestra.run.smithi100.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 388 kB/s | 36 kB 00:00 2024-02-14T17:37:05.215 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:05.231 INFO:teuthology.orchestra.run.smithi100.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 218 kB/s | 20 kB 00:00 2024-02-14T17:37:05.256 INFO:teuthology.orchestra.run.smithi100.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2024-02-14T17:37:05.264 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-14T17:37:05.298 INFO:teuthology.orchestra.run.smithi100.stdout:(36/36): python3-webob-1.8.7-6.el9.noarch.rpm 980 kB/s | 230 kB 00:00 2024-02-14T17:37:05.303 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:05.303 INFO:teuthology.orchestra.run.smithi100.stdout:Total 9.0 MB/s | 30 MB 00:03 2024-02-14T17:37:05.303 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-14T17:37:05.323 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:37:05.333 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-14T17:37:05.368 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:37:05.368 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:37:05.406 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-14T17:37:05.440 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-14T17:37:05.473 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-14T17:37:05.564 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-14T17:37:05.634 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-14T17:37:05.675 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-14T17:37:05.748 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-14T17:37:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:37:05.775 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:37:05.809 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-14T17:37:05.848 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-14T17:37:05.897 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-14T17:37:05.960 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-14T17:37:06.144 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-14T17:37:06.194 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-14T17:37:06.224 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-14T17:37:06.258 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-14T17:37:06.288 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-14T17:37:06.364 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-14T17:37:06.399 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-14T17:37:06.432 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-14T17:37:06.472 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-14T17:37:06.507 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-14T17:37:06.542 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-14T17:37:06.632 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-14T17:37:06.682 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-14T17:37:06.753 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-14T17:37:06.892 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-14T17:37:07.041 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-14T17:37:07.104 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:37:07.168 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-14T17:37:07.197 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9. 31/36 2024-02-14T17:37:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-14T17:37:07.219 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 32/36 2024-02-14T17:37:07.245 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 32/36 2024-02-14T17:37:07.245 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:07.245 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-14T17:37:07.245 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T17:37:07.245 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T17:37:07.245 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-14T17:37:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-14T17:37:07.361 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-14T17:37:07.394 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-14T17:37:07.486 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-14T17:37:07.571 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-14T17:37:07.613 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-14T17:37:07.684 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-14T17:37:07.738 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-14T17:37:07.785 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-14T17:37:07.834 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-14T17:37:07.890 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-14T17:37:08.073 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-14T17:37:08.123 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-14T17:37:08.163 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-14T17:37:08.205 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-14T17:37:08.235 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-14T17:37:08.310 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-14T17:37:08.345 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-14T17:37:08.379 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-14T17:37:08.419 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-14T17:37:08.454 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-14T17:37:08.480 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-14T17:37:08.569 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-14T17:37:08.640 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-14T17:37:08.687 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 33/36 2024-02-14T17:37:08.712 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 33/36 2024-02-14T17:37:08.713 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:08.713 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-14T17:37:08.713 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T17:37:08.713 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T17:37:08.713 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:08.725 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-14T17:37:08.869 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-14T17:37:08.987 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-14T17:37:09.146 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9. 31/36 2024-02-14T17:37:09.154 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 34/36 2024-02-14T17:37:09.175 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 32/36 2024-02-14T17:37:09.177 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 34/36 2024-02-14T17:37:09.177 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:09.177 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-14T17:37:09.177 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T17:37:09.177 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T17:37:09.178 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:09.201 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 32/36 2024-02-14T17:37:09.201 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:09.201 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-14T17:37:09.201 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T17:37:09.201 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-14T17:37:09.201 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:09.388 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 35/36 2024-02-14T17:37:09.411 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 35/36 2024-02-14T17:37:09.411 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:09.412 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-14T17:37:09.412 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T17:37:09.412 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T17:37:09.412 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:09.477 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 36/36 2024-02-14T17:37:10.655 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 33/36 2024-02-14T17:37:10.679 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 33/36 2024-02-14T17:37:10.679 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:10.679 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-14T17:37:10.680 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T17:37:10.680 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-14T17:37:10.680 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:11.129 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 34/36 2024-02-14T17:37:11.152 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 34/36 2024-02-14T17:37:11.152 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:11.152 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-14T17:37:11.152 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T17:37:11.152 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-14T17:37:11.152 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:11.288 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 36/36 2024-02-14T17:37:11.288 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 1/36 2024-02-14T17:37:11.288 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 2/36 2024-02-14T17:37:11.288 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 3/36 2024-02-14T17:37:11.288 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 4/36 2024-02-14T17:37:11.288 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 5/36 2024-02-14T17:37:11.288 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9. 6/36 2024-02-14T17:37:11.288 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-14T17:37:11.288 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-14T17:37:11.288 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-14T17:37:11.289 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-14T17:37:11.290 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-14T17:37:11.362 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 35/36 2024-02-14T17:37:11.386 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 35/36 2024-02-14T17:37:11.386 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:11.387 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-14T17:37:11.387 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T17:37:11.387 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-14T17:37:11.387 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 36/36 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: lua-5.4.4-4.el9.x86_64 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-14T17:37:12.215 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-14T17:37:12.216 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply-3.11-14.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:12.217 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:12.556 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-base 2024-02-14T17:37:13.128 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:13 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:13.215 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 36/36 2024-02-14T17:37:13.215 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 1/36 2024-02-14T17:37:13.215 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 2/36 2024-02-14T17:37:13.215 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 3/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 4/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 5/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9. 6/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-14T17:37:13.216 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-14T17:37:13.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-14T17:37:13.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-14T17:37:13.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-14T17:37:13.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-14T17:37:13.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-14T17:37:13.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-14T17:37:13.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-14T17:37:13.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-14T17:37:13.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-14T17:37:13.217 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-14T17:37:13.218 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-14T17:37:13.218 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-14T17:37:13.218 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-14T17:37:13.218 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-14T17:37:13.219 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-14T17:37:13.219 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-14T17:37:13.219 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-14T17:37:13.219 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-14T17:37:13.243 INFO:teuthology.orchestra.run.smithi179.stdout:Package ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:37:13.295 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:13.296 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-02-14T17:37:13.296 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:13.357 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install cephadm 2024-02-14T17:37:13.931 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:13 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:14.096 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 748 k 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:14.097 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 748 k 2024-02-14T17:37:14.098 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 754 k 2024-02-14T17:37:14.098 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-14T17:37:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:14.266 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-14T17:37:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-14T17:37:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-14T17:37:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-14T17:37:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-14T17:37:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-14T17:37:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-14T17:37:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-14T17:37:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-14T17:37:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-14T17:37:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-14T17:37:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:14.269 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:14.636 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-base 2024-02-14T17:37:15.216 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:13 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:15.246 INFO:teuthology.orchestra.run.smithi179.stdout:cephadm-19.0.0-1430.g8727ce93.el9.noarch.rpm 651 kB/s | 748 kB 00:01 2024-02-14T17:37:15.247 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:15.248 INFO:teuthology.orchestra.run.smithi179.stdout:Total 650 kB/s | 748 kB 00:01 2024-02-14T17:37:15.248 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:15.254 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:15.254 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:15.260 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:15.260 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:15.330 INFO:teuthology.orchestra.run.smithi100.stdout:Package ceph-base-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:37:15.336 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:15.381 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:15.382 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-02-14T17:37:15.382 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:15.441 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install cephadm 2024-02-14T17:37:15.756 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:37:15.854 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:37:16.007 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:14 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:16.170 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:16.171 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:37:16.171 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:16.171 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:37:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 748 k 2024-02-14T17:37:16.171 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:16.171 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:37:16.171 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:16.172 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-02-14T17:37:16.172 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:16.172 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 748 k 2024-02-14T17:37:16.172 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 754 k 2024-02-14T17:37:16.172 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:37:16.260 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:37:16.552 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:37:16.552 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:16.553 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:37:16.553 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:16.553 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:16.553 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:16.747 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-immutable-object-cache 2024-02-14T17:37:17.153 INFO:teuthology.orchestra.run.smithi100.stdout:cephadm-19.0.0-1430.g8727ce93.el9.noarch.rpm 762 kB/s | 748 kB 00:00 2024-02-14T17:37:17.154 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:17.154 INFO:teuthology.orchestra.run.smithi100.stdout:Total 761 kB/s | 748 kB 00:00 2024-02-14T17:37:17.154 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:37:17.160 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:37:17.160 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:37:17.165 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:37:17.166 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:37:17.239 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:37:17.329 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:17 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:17.499 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:17.500 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:17.500 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-02-14T17:37:17.500 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:17.500 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:17.500 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 145 k 2024-02-14T17:37:17.500 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:17.500 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:17.501 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:17.501 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-02-14T17:37:17.501 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:17.501 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 145 k 2024-02-14T17:37:17.501 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 436 k 2024-02-14T17:37:17.501 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:17.660 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:37:17.749 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:37:18.137 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:37:18.186 INFO:teuthology.orchestra.run.smithi179.stdout:ceph-immutable-object-cache-19.0.0-1430.g8727ce 212 kB/s | 145 kB 00:00 2024-02-14T17:37:18.187 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:18.187 INFO:teuthology.orchestra.run.smithi179.stdout:Total 211 kB/s | 145 kB 00:00 2024-02-14T17:37:18.187 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:18.195 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:18.195 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:18.248 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:18.248 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:18.401 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:18.462 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:37:18.471 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/1 2024-02-14T17:37:18.471 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:18.471 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:18.471 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:18.471 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:18.471 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:18.486 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:37:18.486 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:18.486 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-14T17:37:18.486 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:18.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-immutable-object-cache 2024-02-14T17:37:19.255 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:17 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:19.421 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:19.422 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:19.422 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repo Size 2024-02-14T17:37:19.422 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:19.422 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:37:19.422 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 145 k 2024-02-14T17:37:19.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:19.423 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:37:19.423 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:19.423 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-02-14T17:37:19.423 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:19.423 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 145 k 2024-02-14T17:37:19.423 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 436 k 2024-02-14T17:37:19.423 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:37:19.454 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:37:19.454 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:19.454 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:37:19.454 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:19.454 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:19.454 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:19.682 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr 2024-02-14T17:37:19.993 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-immutable-object-cache-19.0.0-1430.g8727ce 255 kB/s | 145 kB 00:00 2024-02-14T17:37:19.994 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:19.994 INFO:teuthology.orchestra.run.smithi100.stdout:Total 254 kB/s | 145 kB 00:00 2024-02-14T17:37:19.994 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:37:20.001 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:37:20.001 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:37:20.053 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:37:20.053 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:37:20.205 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:37:20.269 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:37:20.273 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:20 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:20.292 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:37:20.292 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:20.292 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-14T17:37:20.292 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:20.386 INFO:teuthology.orchestra.run.smithi179.stdout:Package ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:37:20.438 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:20.439 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-02-14T17:37:20.440 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:20.499 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-dashboard 2024-02-14T17:37:21.077 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:21 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:21.243 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:21.244 INFO:teuthology.orchestra.run.smithi179.stdout:====================================================================================== 2024-02-14T17:37:21.244 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:37:21.244 INFO:teuthology.orchestra.run.smithi179.stdout:====================================================================================== 2024-02-14T17:37:21.244 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:21.244 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 1.8 M 2024-02-14T17:37:21.244 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-02-14T17:37:21.244 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-14T17:37:21.244 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 31 k 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 15 k 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout:====================================================================================== 2024-02-14T17:37:21.245 INFO:teuthology.orchestra.run.smithi179.stdout:Install 12 Packages 2024-02-14T17:37:21.246 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:21.246 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 6.9 M 2024-02-14T17:37:21.246 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 26 M 2024-02-14T17:37:21.246 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93. 1/1 2024-02-14T17:37:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:21.290 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:21.290 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:21.290 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:21.555 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-mgr 2024-02-14T17:37:21.832 INFO:teuthology.orchestra.run.smithi179.stdout:(1/12): ceph-prometheus-alerts-19.0.0-1430.g872 39 kB/s | 15 kB 00:00 2024-02-14T17:37:21.907 INFO:teuthology.orchestra.run.smithi179.stdout:(2/12): ceph-grafana-dashboards-19.0.0-1430.g87 67 kB/s | 31 kB 00:00 2024-02-14T17:37:22.116 INFO:teuthology.orchestra.run.smithi179.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-02-14T17:37:22.140 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:20 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:22.141 INFO:teuthology.orchestra.run.smithi179.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.0 MB/s | 238 kB 00:00 2024-02-14T17:37:22.252 INFO:teuthology.orchestra.run.smithi100.stdout:Package ceph-mgr-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:37:22.302 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:22.304 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-02-14T17:37:22.304 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:22.349 INFO:teuthology.orchestra.run.smithi179.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.6 MB/s | 863 kB 00:00 2024-02-14T17:37:22.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-mgr-dashboard 2024-02-14T17:37:22.633 INFO:teuthology.orchestra.run.smithi179.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 68 kB/s | 19 kB 00:00 2024-02-14T17:37:22.825 INFO:teuthology.orchestra.run.smithi179.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 801 kB/s | 548 kB 00:00 2024-02-14T17:37:22.941 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:20 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:22.950 INFO:teuthology.orchestra.run.smithi179.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.1 MB/s | 144 kB 00:00 2024-02-14T17:37:22.992 INFO:teuthology.orchestra.run.smithi179.stdout:(9/12): ceph-mgr-dashboard-19.0.0-1430.g8727ce9 1.1 MB/s | 1.8 MB 00:01 2024-02-14T17:37:23.078 INFO:teuthology.orchestra.run.smithi179.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 242 kB/s | 31 kB 00:00 2024-02-14T17:37:23.107 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:23.108 INFO:teuthology.orchestra.run.smithi100.stdout:====================================================================================== 2024-02-14T17:37:23.108 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:37:23.108 INFO:teuthology.orchestra.run.smithi100.stdout:====================================================================================== 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 1.8 M 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 31 k 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 15 k 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-14T17:37:23.109 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-14T17:37:23.110 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-14T17:37:23.110 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:23.110 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:37:23.110 INFO:teuthology.orchestra.run.smithi100.stdout:====================================================================================== 2024-02-14T17:37:23.110 INFO:teuthology.orchestra.run.smithi100.stdout:Install 12 Packages 2024-02-14T17:37:23.110 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:23.111 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 6.9 M 2024-02-14T17:37:23.111 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 26 M 2024-02-14T17:37:23.111 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:37:23.360 INFO:teuthology.orchestra.run.smithi179.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.8 MB/s | 2.0 MB 00:00 2024-02-14T17:37:23.577 INFO:teuthology.orchestra.run.smithi179.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 323 kB/s | 188 kB 00:00 2024-02-14T17:37:23.580 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:23.580 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.9 MB/s | 6.9 MB 00:02 2024-02-14T17:37:23.600 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:23.619 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:23.619 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:23.800 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:23.800 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:23.864 INFO:teuthology.orchestra.run.smithi100.stdout:(1/12): ceph-grafana-dashboards-19.0.0-1430.g87 73 kB/s | 31 kB 00:00 2024-02-14T17:37:23.889 INFO:teuthology.orchestra.run.smithi100.stdout:(2/12): ceph-prometheus-alerts-19.0.0-1430.g872 34 kB/s | 15 kB 00:00 2024-02-14T17:37:24.056 INFO:teuthology.orchestra.run.smithi100.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.4 MB/s | 238 kB 00:00 2024-02-14T17:37:24.098 INFO:teuthology.orchestra.run.smithi100.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-02-14T17:37:24.311 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:24.377 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-14T17:37:24.412 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-14T17:37:24.456 INFO:teuthology.orchestra.run.smithi100.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.5 MB/s | 548 kB 00:00 2024-02-14T17:37:24.459 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-14T17:37:24.498 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-14T17:37:24.506 INFO:teuthology.orchestra.run.smithi100.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-02-14T17:37:24.601 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-14T17:37:24.657 INFO:teuthology.orchestra.run.smithi100.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 14 MB/s | 2.0 MB 00:00 2024-02-14T17:37:24.716 INFO:teuthology.orchestra.run.smithi100.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.4 MB/s | 144 kB 00:00 2024-02-14T17:37:24.731 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-14T17:37:24.766 INFO:teuthology.orchestra.run.smithi100.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 614 kB/s | 31 kB 00:00 2024-02-14T17:37:24.818 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-14T17:37:24.824 INFO:teuthology.orchestra.run.smithi100.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-02-14T17:37:24.860 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-14T17:37:24.897 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-14T17:37:24.929 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9 10/12 2024-02-14T17:37:25.169 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el 11/12 2024-02-14T17:37:25.183 INFO:teuthology.orchestra.run.smithi100.stdout:(11/12): ceph-mgr-dashboard-19.0.0-1430.g8727ce 1.0 MB/s | 1.8 MB 00:01 2024-02-14T17:37:25.196 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 12/12 2024-02-14T17:37:25.233 INFO:teuthology.orchestra.run.smithi100.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 734 kB/s | 863 kB 00:01 2024-02-14T17:37:25.237 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:25.238 INFO:teuthology.orchestra.run.smithi100.stdout:Total 3.2 MB/s | 6.9 MB 00:02 2024-02-14T17:37:25.255 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:37:25.272 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:37:25.272 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:37:25.460 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:37:25.460 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:37:25.972 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 12/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el 1/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 2/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9 3/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-14T17:37:26.013 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-14T17:37:26.050 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-14T17:37:26.094 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-14T17:37:26.141 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-14T17:37:26.179 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-14T17:37:26.284 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-14T17:37:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-14T17:37:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:26.394 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:37:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-14T17:37:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-14T17:37:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-14T17:37:26.394 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-14T17:37:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-14T17:37:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-14T17:37:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-14T17:37:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-14T17:37:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-14T17:37:26.395 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:26.395 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:26.414 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-14T17:37:26.499 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-14T17:37:26.571 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-14T17:37:26.613 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-14T17:37:26.631 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-14T17:37:26.644 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9 10/12 2024-02-14T17:37:26.884 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el 11/12 2024-02-14T17:37:26.919 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 12/12 2024-02-14T17:37:27.219 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:27 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:27.397 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:27.398 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================ 2024-02-14T17:37:27.398 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:37:27.398 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================ 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 7.4 M 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-14T17:37:27.399 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-14T17:37:27.400 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-14T17:37:27.400 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:27.400 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:27.400 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================ 2024-02-14T17:37:27.400 INFO:teuthology.orchestra.run.smithi179.stdout:Install 12 Packages 2024-02-14T17:37:27.400 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:27.400 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 37 M 2024-02-14T17:37:27.401 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 209 M 2024-02-14T17:37:27.401 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:27.584 INFO:teuthology.orchestra.run.smithi179.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 278 kB/s | 32 kB 00:00 2024-02-14T17:37:27.617 INFO:teuthology.orchestra.run.smithi179.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 520 kB/s | 17 kB 00:00 2024-02-14T17:37:27.651 INFO:teuthology.orchestra.run.smithi179.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-02-14T17:37:27.752 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 12/12 2024-02-14T17:37:27.752 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el 1/12 2024-02-14T17:37:27.752 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noa 2/12 2024-02-14T17:37:27.752 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9 3/12 2024-02-14T17:37:27.752 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-14T17:37:27.752 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-14T17:37:27.753 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-14T17:37:27.753 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-14T17:37:27.753 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-14T17:37:27.753 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-14T17:37:27.753 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-14T17:37:27.753 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-14T17:37:27.801 INFO:teuthology.orchestra.run.smithi179.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 32 MB/s | 4.7 MB 00:00 2024-02-14T17:37:27.943 INFO:teuthology.orchestra.run.smithi179.stdout:(5/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-02-14T17:37:28.010 INFO:teuthology.orchestra.run.smithi179.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.5 MB/s | 3.0 MB 00:00 2024-02-14T17:37:28.027 INFO:teuthology.orchestra.run.smithi179.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.6 MB/s | 217 kB 00:00 2024-02-14T17:37:28.121 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-14T17:37:28.121 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:28.121 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:28.121 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:28.122 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:28.144 INFO:teuthology.orchestra.run.smithi179.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 6.8 MB/s | 802 kB 00:00 2024-02-14T17:37:28.394 INFO:teuthology.orchestra.run.smithi179.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-02-14T17:37:28.405 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-14T17:37:28.419 INFO:teuthology.orchestra.run.smithi179.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 711 kB/s | 194 kB 00:00 2024-02-14T17:37:28.561 INFO:teuthology.orchestra.run.smithi179.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-02-14T17:37:28.984 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:26 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:29.158 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:29.159 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-02-14T17:37:29.159 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:37:29.159 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-02-14T17:37:29.159 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:37:29.159 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 7.4 M 2024-02-14T17:37:29.159 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-14T17:37:29.159 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-14T17:37:29.159 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-14T17:37:29.159 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout:Install 12 Packages 2024-02-14T17:37:29.160 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:29.161 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 37 M 2024-02-14T17:37:29.161 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 209 M 2024-02-14T17:37:29.161 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:37:29.583 INFO:teuthology.orchestra.run.smithi100.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 205 kB/s | 32 kB 00:00 2024-02-14T17:37:29.616 INFO:teuthology.orchestra.run.smithi100.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 517 kB/s | 17 kB 00:00 2024-02-14T17:37:29.658 INFO:teuthology.orchestra.run.smithi100.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 844 kB/s | 34 kB 00:00 2024-02-14T17:37:29.842 INFO:teuthology.orchestra.run.smithi100.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2024-02-14T17:37:29.967 INFO:teuthology.orchestra.run.smithi100.stdout:(5/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-02-14T17:37:30.009 INFO:teuthology.orchestra.run.smithi100.stdout:(6/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 5.2 MB/s | 217 kB 00:00 2024-02-14T17:37:30.134 INFO:teuthology.orchestra.run.smithi100.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.2 MB/s | 3.0 MB 00:00 2024-02-14T17:37:30.178 INFO:teuthology.orchestra.run.smithi179.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-1 2.7 MB/s | 7.4 MB 00:02 2024-02-14T17:37:30.181 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:30.182 INFO:teuthology.orchestra.run.smithi179.stdout:Total 13 MB/s | 37 MB 00:02 2024-02-14T17:37:30.196 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:30.223 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:30.223 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:30.368 INFO:teuthology.orchestra.run.smithi100.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-02-14T17:37:30.401 INFO:teuthology.orchestra.run.smithi100.stdout:(9/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 802 kB 00:00 2024-02-14T17:37:30.427 INFO:teuthology.orchestra.run.smithi100.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 194 kB 00:00 2024-02-14T17:37:30.571 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:30.571 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:30.627 INFO:teuthology.orchestra.run.smithi100.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-02-14T17:37:31.265 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:31.342 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-14T17:37:31.374 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-14T17:37:31.411 INFO:teuthology.orchestra.run.smithi100.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-1 3.7 MB/s | 7.4 MB 00:01 2024-02-14T17:37:31.414 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:31.414 INFO:teuthology.orchestra.run.smithi100.stdout:Total 16 MB/s | 37 MB 00:02 2024-02-14T17:37:31.429 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:37:31.455 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:37:31.455 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:37:31.466 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-14T17:37:31.503 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-14T17:37:31.682 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-14T17:37:31.710 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-14T17:37:31.788 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-14T17:37:31.814 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:37:31.815 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:37:32.450 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-14T17:37:32.518 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:37:32.555 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-14T17:37:32.600 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-14T17:37:32.681 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-14T17:37:32.783 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-14T17:37:32.835 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-14T17:37:33.014 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-14T17:37:33.051 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-14T17:37:33.131 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-14T17:37:33.786 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-14T17:37:33.792 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-14T17:37:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-14T17:37:34.816 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-14T17:37:34.843 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 12/12 2024-02-14T17:37:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-14T17:37:36.166 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-14T17:37:36.192 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 12/12 2024-02-14T17:37:37.568 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 12/12 2024-02-14T17:37:37.568 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 1/12 2024-02-14T17:37:37.568 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-14T17:37:37.569 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-14T17:37:37.569 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-14T17:37:37.569 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-14T17:37:37.569 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-14T17:37:37.569 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-14T17:37:37.569 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-14T17:37:37.569 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-14T17:37:37.569 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-14T17:37:37.569 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-14T17:37:38.003 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-14T17:37:38.004 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-14T17:37:38.005 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-14T17:37:38.005 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:38.005 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:38.423 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-rook 2024-02-14T17:37:38.928 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 12/12 2024-02-14T17:37:38.928 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727c 1/12 2024-02-14T17:37:38.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-14T17:37:38.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-14T17:37:38.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-14T17:37:38.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-14T17:37:38.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-14T17:37:38.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-14T17:37:38.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-14T17:37:38.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-14T17:37:38.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-14T17:37:38.929 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-14T17:37:39.007 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:39 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:39.175 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:39.176 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================== 2024-02-14T17:37:39.176 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:37:39.176 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================== 2024-02-14T17:37:39.176 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:39.176 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 52 k 2024-02-14T17:37:39.176 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-02-14T17:37:39.176 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-14T17:37:39.176 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-14T17:37:39.176 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-14T17:37:39.176 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================== 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout:Install 13 Packages 2024-02-14T17:37:39.177 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:39.178 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 2.0 M 2024-02-14T17:37:39.178 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 26 M 2024-02-14T17:37:39.178 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:39.363 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-14T17:37:39.363 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:39.363 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:39.363 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:39.363 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-14T17:37:39.363 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-14T17:37:39.363 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-14T17:37:39.364 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-14T17:37:39.364 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-14T17:37:39.364 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-14T17:37:39.364 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-14T17:37:39.364 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-14T17:37:39.364 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-14T17:37:39.364 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-14T17:37:39.364 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-14T17:37:39.364 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:39.365 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:39.527 INFO:teuthology.orchestra.run.smithi179.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 153 kB/s | 24 kB 00:00 2024-02-14T17:37:39.552 INFO:teuthology.orchestra.run.smithi179.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 98 kB/s | 18 kB 00:00 2024-02-14T17:37:39.644 INFO:teuthology.orchestra.run.smithi179.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 174 kB 00:00 2024-02-14T17:37:39.669 INFO:teuthology.orchestra.run.smithi179.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 132 kB 00:00 2024-02-14T17:37:39.751 INFO:teuthology.orchestra.run.smithi179.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.1 MB/s | 211 kB 00:00 2024-02-14T17:37:39.751 INFO:teuthology.orchestra.run.smithi179.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 639 kB/s | 43 kB 00:00 2024-02-14T17:37:39.770 INFO:teuthology.orchestra.run.smithi179.stdout:(7/13): ceph-mgr-rook-19.0.0-1430.g8727ce93.el9 130 kB/s | 52 kB 00:00 2024-02-14T17:37:39.778 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-mgr-rook 2024-02-14T17:37:39.937 INFO:teuthology.orchestra.run.smithi179.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 143 kB/s | 32 kB 00:00 2024-02-14T17:37:39.962 INFO:teuthology.orchestra.run.smithi179.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 66 kB/s | 15 kB 00:00 2024-02-14T17:37:40.071 INFO:teuthology.orchestra.run.smithi179.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 675 kB/s | 203 kB 00:00 2024-02-14T17:37:40.112 INFO:teuthology.orchestra.run.smithi179.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-02-14T17:37:40.138 INFO:teuthology.orchestra.run.smithi179.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 336 kB/s | 59 kB 00:00 2024-02-14T17:37:40.154 INFO:teuthology.orchestra.run.smithi179.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 1.1 MB/s | 90 kB 00:00 2024-02-14T17:37:40.159 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:40.159 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.1 MB/s | 2.0 MB 00:00 2024-02-14T17:37:40.159 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:40.172 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:40.172 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:40.270 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:40.270 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:40.372 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:38 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:40.537 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:40.538 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================== 2024-02-14T17:37:40.538 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:37:40.538 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================== 2024-02-14T17:37:40.538 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 52 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-14T17:37:40.539 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-14T17:37:40.540 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:40.540 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:37:40.540 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================== 2024-02-14T17:37:40.540 INFO:teuthology.orchestra.run.smithi100.stdout:Install 13 Packages 2024-02-14T17:37:40.540 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:40.540 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 2.0 M 2024-02-14T17:37:40.540 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 26 M 2024-02-14T17:37:40.541 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:37:40.811 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:40.969 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-14T17:37:41.001 INFO:teuthology.orchestra.run.smithi100.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 112 kB/s | 24 kB 00:00 2024-02-14T17:37:41.024 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-14T17:37:41.026 INFO:teuthology.orchestra.run.smithi100.stdout:(2/13): ceph-mgr-rook-19.0.0-1430.g8727ce93.el9 214 kB/s | 52 kB 00:00 2024-02-14T17:37:41.060 INFO:teuthology.orchestra.run.smithi100.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 65 kB/s | 18 kB 00:00 2024-02-14T17:37:41.079 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-14T17:37:41.117 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-14T17:37:41.159 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-14T17:37:41.194 INFO:teuthology.orchestra.run.smithi100.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 913 kB/s | 174 kB 00:00 2024-02-14T17:37:41.251 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-14T17:37:41.319 INFO:teuthology.orchestra.run.smithi100.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 453 kB/s | 132 kB 00:00 2024-02-14T17:37:41.351 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-14T17:37:41.361 INFO:teuthology.orchestra.run.smithi100.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 700 kB/s | 211 kB 00:00 2024-02-14T17:37:41.378 INFO:teuthology.orchestra.run.smithi100.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 233 kB/s | 43 kB 00:00 2024-02-14T17:37:41.401 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-14T17:37:41.920 INFO:teuthology.orchestra.run.smithi100.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 54 kB/s | 32 kB 00:00 2024-02-14T17:37:41.945 INFO:teuthology.orchestra.run.smithi100.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 26 kB/s | 15 kB 00:00 2024-02-14T17:37:42.120 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-14T17:37:42.145 INFO:teuthology.orchestra.run.smithi100.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 295 kB/s | 59 kB 00:00 2024-02-14T17:37:42.165 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-14T17:37:42.199 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-14T17:37:42.247 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-14T17:37:42.254 INFO:teuthology.orchestra.run.smithi100.stdout:(11/13): python3-google-auth-2.27.0-1.el9.noarc 231 kB/s | 203 kB 00:00 2024-02-14T17:37:42.269 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 13/13 2024-02-14T17:37:42.338 INFO:teuthology.orchestra.run.smithi100.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 468 kB/s | 90 kB 00:00 2024-02-14T17:37:43.061 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 13/13 2024-02-14T17:37:43.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 1/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-14T17:37:43.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-14T17:37:43.354 INFO:teuthology.orchestra.run.smithi100.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 729 kB/s | 1.0 MB 00:01 2024-02-14T17:37:43.360 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:43.360 INFO:teuthology.orchestra.run.smithi100.stdout:Total 744 kB/s | 2.0 MB 00:02 2024-02-14T17:37:43.360 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:37:43.374 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:37:43.375 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-14T17:37:43.426 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-14T17:37:43.427 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-14T17:37:43.427 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-14T17:37:43.427 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-14T17:37:43.427 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-14T17:37:43.427 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-14T17:37:43.427 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:43.427 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:43.479 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:37:43.481 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:37:43.639 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-mgr-cephadm 2024-02-14T17:37:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:37:44.168 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-14T17:37:44.226 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:44 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:44.229 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-14T17:37:44.285 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-14T17:37:44.315 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-14T17:37:44.364 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-14T17:37:44.393 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:44.394 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================== 2024-02-14T17:37:44.394 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:37:44.394 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================== 2024-02-14T17:37:44.394 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:44.394 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 138 k 2024-02-14T17:37:44.394 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-02-14T17:37:44.394 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-14T17:37:44.394 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-14T17:37:44.394 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-14T17:37:44.395 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-14T17:37:44.395 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-14T17:37:44.395 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:44.395 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:44.395 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================== 2024-02-14T17:37:44.395 INFO:teuthology.orchestra.run.smithi179.stdout:Install 6 Packages 2024-02-14T17:37:44.395 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:44.396 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 6.8 M 2024-02-14T17:37:44.396 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 33 M 2024-02-14T17:37:44.396 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:44.465 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-14T17:37:44.574 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-14T17:37:44.632 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-14T17:37:44.650 INFO:teuthology.orchestra.run.smithi179.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.0 MB/s | 227 kB 00:00 2024-02-14T17:37:44.676 INFO:teuthology.orchestra.run.smithi179.stdout:(2/6): ceph-mgr-cephadm-19.0.0-1430.g8727ce93.e 580 kB/s | 138 kB 00:00 2024-02-14T17:37:44.834 INFO:teuthology.orchestra.run.smithi179.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 5.8 MB 00:00 2024-02-14T17:37:45.346 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-14T17:37:45.388 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-14T17:37:45.432 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-14T17:37:45.479 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-14T17:37:45.501 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 13/13 2024-02-14T17:37:45.776 INFO:teuthology.orchestra.run.smithi179.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 52 kB/s | 58 kB 00:01 2024-02-14T17:37:45.935 INFO:teuthology.orchestra.run.smithi179.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 46 kB/s | 51 kB 00:01 2024-02-14T17:37:46.303 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 13/13 2024-02-14T17:37:46.303 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 1/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-14T17:37:46.304 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-14T17:37:46.477 INFO:teuthology.orchestra.run.smithi179.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 300 kB/s | 548 kB 00:01 2024-02-14T17:37:46.483 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:46.484 INFO:teuthology.orchestra.run.smithi179.stdout:Total 3.2 MB/s | 6.8 MB 00:02 2024-02-14T17:37:46.484 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:46.493 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:46.493 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:46.588 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:46.590 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:46.666 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-14T17:37:46.666 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:46.666 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:46.666 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:46.666 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-14T17:37:46.666 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-14T17:37:46.666 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-14T17:37:46.666 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-14T17:37:46.666 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-14T17:37:46.666 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-14T17:37:46.667 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-14T17:37:46.667 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-14T17:37:46.667 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-14T17:37:46.667 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-14T17:37:46.667 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-14T17:37:46.667 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-14T17:37:46.667 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:46.667 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:46.885 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:46.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-mgr-cephadm 2024-02-14T17:37:47.060 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-14T17:37:47.103 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-14T17:37:47.477 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:45 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:47.610 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-14T17:37:47.642 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:47.642 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================== 2024-02-14T17:37:47.642 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================== 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 138 k 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout:======================================================================================== 2024-02-14T17:37:47.643 INFO:teuthology.orchestra.run.smithi100.stdout:Install 6 Packages 2024-02-14T17:37:47.644 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:47.644 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 6.8 M 2024-02-14T17:37:47.644 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 33 M 2024-02-14T17:37:47.644 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:37:47.681 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-14T17:37:47.742 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-14T17:37:47.766 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 6/6 2024-02-14T17:37:47.873 INFO:teuthology.orchestra.run.smithi100.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 227 kB 00:00 2024-02-14T17:37:47.957 INFO:teuthology.orchestra.run.smithi100.stdout:(2/6): ceph-mgr-cephadm-19.0.0-1430.g8727ce93.e 582 kB/s | 138 kB 00:00 2024-02-14T17:37:48.065 INFO:teuthology.orchestra.run.smithi100.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2024-02-14T17:37:48.299 INFO:teuthology.orchestra.run.smithi100.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 182 kB/s | 58 kB 00:00 2024-02-14T17:37:48.374 INFO:teuthology.orchestra.run.smithi100.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 179 kB/s | 51 kB 00:00 2024-02-14T17:37:48.399 INFO:teuthology.orchestra.run.smithi100.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2024-02-14T17:37:48.403 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:48.403 INFO:teuthology.orchestra.run.smithi100.stdout:Total 8.9 MB/s | 6.8 MB 00:00 2024-02-14T17:37:48.403 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:37:48.412 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:37:48.412 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:37:48.503 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:37:48.503 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:37:48.563 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 6/6 2024-02-14T17:37:48.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/6 2024-02-14T17:37:48.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-14T17:37:48.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-14T17:37:48.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-14T17:37:48.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-14T17:37:48.806 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:37:48.883 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-14T17:37:48.883 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:48.883 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:37:48.883 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:48.883 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-14T17:37:48.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-14T17:37:48.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-14T17:37:48.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-14T17:37:48.884 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-14T17:37:48.884 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:48.884 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:48.973 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-14T17:37:49.026 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-14T17:37:49.116 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-fuse 2024-02-14T17:37:49.538 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-14T17:37:49.640 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-14T17:37:49.697 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:49 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:49.707 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-14T17:37:49.730 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 6/6 2024-02-14T17:37:49.865 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:49.866 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:49.866 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:37:49.866 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:49.866 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:49.866 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 886 k 2024-02-14T17:37:49.866 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-02-14T17:37:49.866 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-14T17:37:49.867 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:49.867 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:49.867 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:49.867 INFO:teuthology.orchestra.run.smithi179.stdout:Install 2 Packages 2024-02-14T17:37:49.867 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:49.867 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 965 k 2024-02-14T17:37:49.867 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 2.8 M 2024-02-14T17:37:49.867 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:49.987 INFO:teuthology.orchestra.run.smithi179.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 663 kB/s | 79 kB 00:00 2024-02-14T17:37:50.312 INFO:teuthology.orchestra.run.smithi179.stdout:(2/2): ceph-fuse-19.0.0-1430.g8727ce93.el9.x86_ 1.9 MB/s | 886 kB 00:00 2024-02-14T17:37:50.313 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:50.313 INFO:teuthology.orchestra.run.smithi179.stdout:Total 2.1 MB/s | 965 kB 00:00 2024-02-14T17:37:50.313 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:50.333 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:50.333 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:50.410 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:50.411 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:50.526 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 6/6 2024-02-14T17:37:50.526 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 1/6 2024-02-14T17:37:50.526 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-14T17:37:50.526 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-14T17:37:50.526 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-14T17:37:50.526 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-14T17:37:50.590 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:50.738 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-14T17:37:50.803 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2/2 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:50.833 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:51.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-fuse 2024-02-14T17:37:51.529 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2/2 2024-02-14T17:37:51.529 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/2 2024-02-14T17:37:51.647 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:49 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:51.814 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:51.814 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:51.814 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:37:51.814 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:51.814 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:37:51.814 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 886 k 2024-02-14T17:37:51.814 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-14T17:37:51.815 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-14T17:37:51.815 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:51.815 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:37:51.815 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:51.815 INFO:teuthology.orchestra.run.smithi100.stdout:Install 2 Packages 2024-02-14T17:37:51.815 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:51.815 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 965 k 2024-02-14T17:37:51.815 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 2.8 M 2024-02-14T17:37:51.816 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:37:51.839 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-14T17:37:51.839 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:51.839 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:37:51.839 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-14T17:37:51.839 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:51.839 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:51.926 INFO:teuthology.orchestra.run.smithi100.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 714 kB/s | 79 kB 00:00 2024-02-14T17:37:52.140 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-volume 2024-02-14T17:37:52.618 INFO:teuthology.orchestra.run.smithi100.stdout:(2/2): ceph-fuse-19.0.0-1430.g8727ce93.el9.x86_ 1.1 MB/s | 886 kB 00:00 2024-02-14T17:37:52.619 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:52.619 INFO:teuthology.orchestra.run.smithi100.stdout:Total 1.2 MB/s | 965 kB 00:00 2024-02-14T17:37:52.620 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:37:52.640 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:37:52.640 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:37:52.718 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:37:52.718 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:37:52.731 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:52 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:52.899 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:37:52.900 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:52.900 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:52.900 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-02-14T17:37:52.900 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:52.900 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:52.900 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 264 k 2024-02-14T17:37:52.900 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-02-14T17:37:52.901 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-14T17:37:52.901 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:52.901 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:52.901 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:52.901 INFO:teuthology.orchestra.run.smithi179.stdout:Install 2 Packages 2024-02-14T17:37:52.901 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:52.901 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 332 k 2024-02-14T17:37:52.902 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 1.5 M 2024-02-14T17:37:52.902 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:53.045 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-14T17:37:53.051 INFO:teuthology.orchestra.run.smithi179.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 466 kB/s | 69 kB 00:00 2024-02-14T17:37:53.120 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2/2 2024-02-14T17:37:53.175 INFO:teuthology.orchestra.run.smithi179.stdout:(2/2): ceph-volume-19.0.0-1430.g8727ce93.el9.no 967 kB/s | 264 kB 00:00 2024-02-14T17:37:53.176 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:53.176 INFO:teuthology.orchestra.run.smithi179.stdout:Total 1.2 MB/s | 332 kB 00:00 2024-02-14T17:37:53.176 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:53.184 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:53.184 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:53.229 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:53.229 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:53.382 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:53.557 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-14T17:37:53.594 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2/2 2024-02-14T17:37:53.609 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2/2 2024-02-14T17:37:53.610 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:53.610 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-14T17:37:53.610 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:53.832 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2/2 2024-02-14T17:37:53.832 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/2 2024-02-14T17:37:54.161 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-14T17:37:54.161 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:54.161 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:54.161 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-14T17:37:54.161 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:54.161 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:54.396 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 1/2 2024-02-14T17:37:54.459 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-volume 2024-02-14T17:37:54.707 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-14T17:37:54.707 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:54.707 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:37:54.707 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:54.707 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-14T17:37:54.707 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:54.707 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:54.989 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librados-devel 2024-02-14T17:37:55.049 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:53 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:55.213 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:55.214 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:55.214 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-02-14T17:37:55.214 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:55.214 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:37:55.214 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.0.0-1430.g8727ce93.el9 ceph-noarch 264 k 2024-02-14T17:37:55.214 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-02-14T17:37:55.214 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-14T17:37:55.214 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:55.214 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:37:55.214 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:55.215 INFO:teuthology.orchestra.run.smithi100.stdout:Install 2 Packages 2024-02-14T17:37:55.215 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:55.215 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 332 k 2024-02-14T17:37:55.215 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 1.5 M 2024-02-14T17:37:55.215 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:37:55.325 INFO:teuthology.orchestra.run.smithi100.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 629 kB/s | 69 kB 00:00 2024-02-14T17:37:55.573 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:55 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:55.740 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:55.740 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 127 k 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 127 k 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 456 k 2024-02-14T17:37:55.741 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:55.808 INFO:teuthology.orchestra.run.smithi100.stdout:(2/2): ceph-volume-19.0.0-1430.g8727ce93.el9.no 445 kB/s | 264 kB 00:00 2024-02-14T17:37:55.809 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:Total 558 kB/s | 332 kB 00:00 2024-02-14T17:37:55.810 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:37:55.818 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:37:55.819 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:37:55.855 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:37:55.856 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:37:55.972 INFO:teuthology.orchestra.run.smithi179.stdout:librados-devel-19.0.0-1430.g8727ce93.el9.x86_64 549 kB/s | 127 kB 00:00 2024-02-14T17:37:55.973 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:55.973 INFO:teuthology.orchestra.run.smithi179.stdout:Total 546 kB/s | 127 kB 00:00 2024-02-14T17:37:55.973 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:55.978 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:55.978 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:56.009 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:37:56.034 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:56.034 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:56.188 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-14T17:37:56.215 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:56.228 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2/2 2024-02-14T17:37:56.243 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2/2 2024-02-14T17:37:56.243 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:37:56.243 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-14T17:37:56.243 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:56.321 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:37:56.774 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:37:56.961 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 1/2 2024-02-14T17:37:57.071 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:37:57.071 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:57.071 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:37:57.071 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:57.071 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:57.072 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:57.289 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-14T17:37:57.289 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:57.289 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:57.289 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.0.0-1430.g8727ce93.el9.noarch 2024-02-14T17:37:57.289 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-14T17:37:57.289 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:57.290 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:37:57.291 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install libcephfs2 2024-02-14T17:37:57.631 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install librados-devel 2024-02-14T17:37:57.894 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:57 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:58.020 INFO:teuthology.orchestra.run.smithi179.stdout:Package libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:37:58.073 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:58.074 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-02-14T17:37:58.075 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:37:58.132 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install libcephfs-devel 2024-02-14T17:37:58.216 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:56 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:37:58.383 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:37:58.383 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repo Size 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 127 k 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 127 k 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 456 k 2024-02-14T17:37:58.384 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:37:58.715 INFO:teuthology.orchestra.run.smithi100.stdout:librados-devel-19.0.0-1430.g8727ce93.el9.x86_64 383 kB/s | 127 kB 00:00 2024-02-14T17:37:58.716 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:58.717 INFO:teuthology.orchestra.run.smithi100.stdout:Total 381 kB/s | 127 kB 00:00 2024-02-14T17:37:58.717 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:37:58.722 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:37:58.722 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:37:58.729 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:01:58 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:37:58.781 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:37:58.781 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:37:58.901 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:37:58.901 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:58.901 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-02-14T17:37:58.901 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:58.901 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:37:58.901 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 31 k 2024-02-14T17:37:58.901 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:58.901 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:37:58.902 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:37:58.902 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-02-14T17:37:58.902 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:37:58.902 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 31 k 2024-02-14T17:37:58.902 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 140 k 2024-02-14T17:37:58.902 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:37:58.970 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:37:59.083 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:37:59.135 INFO:teuthology.orchestra.run.smithi179.stdout:libcephfs-devel-19.0.0-1430.g8727ce93.el9.x86_6 134 kB/s | 31 kB 00:00 2024-02-14T17:37:59.135 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:37:59.136 INFO:teuthology.orchestra.run.smithi179.stdout:Total 133 kB/s | 31 kB 00:00 2024-02-14T17:37:59.136 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:37:59.139 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:37:59.139 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:37:59.169 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:37:59.169 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:37:59.271 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:37:59.367 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:37:59.621 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:37:59.779 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:37:59.966 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:37:59.966 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:59.966 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:37:59.966 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:37:59.966 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:37:59.966 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:00.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:00.064 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:00.064 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:38:00.064 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:38:00.064 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:00.064 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:38:00.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install libcephfs2 2024-02-14T17:38:00.223 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librados2 2024-02-14T17:38:00.789 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:58 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:00.795 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:00 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:38:00.906 INFO:teuthology.orchestra.run.smithi100.stdout:Package libcephfs2-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:00.909 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:00.957 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:00.958 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-02-14T17:38:00.958 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:00.961 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:38:00.962 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-02-14T17:38:00.962 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:38:01.015 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install libcephfs-devel 2024-02-14T17:38:01.020 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install librbd1 2024-02-14T17:38:01.584 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:01:59 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:01.598 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:01 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:38:01.714 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:01.749 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:01.749 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:01.749 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repo Size 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 31 k 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 31 k 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 140 k 2024-02-14T17:38:01.750 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:38:01.765 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:38:01.766 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-02-14T17:38:01.766 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:38:01.828 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rados 2024-02-14T17:38:02.011 INFO:teuthology.orchestra.run.smithi100.stdout:libcephfs-devel-19.0.0-1430.g8727ce93.el9.x86_6 120 kB/s | 31 kB 00:00 2024-02-14T17:38:02.012 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:38:02.012 INFO:teuthology.orchestra.run.smithi100.stdout:Total 119 kB/s | 31 kB 00:00 2024-02-14T17:38:02.012 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:38:02.015 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:38:02.015 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:38:02.046 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:38:02.047 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:38:02.147 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:38:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:02.408 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:02 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:38:02.522 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:02.573 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:38:02.574 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-02-14T17:38:02.574 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:38:02.628 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rgw 2024-02-14T17:38:02.683 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:02.946 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:02.946 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:02.946 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:38:02.946 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:38:02.946 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:02.946 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:03.128 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install librados2 2024-02-14T17:38:03.202 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:03 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:38:03.315 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:03.368 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:38:03.370 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-02-14T17:38:03.370 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:38:03.440 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-cephfs 2024-02-14T17:38:03.695 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:01 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:03.808 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:03.859 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:03.860 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-02-14T17:38:03.860 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:03.930 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install librbd1 2024-02-14T17:38:04.013 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:04 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:38:04.126 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:04.177 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:38:04.178 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-02-14T17:38:04.178 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:38:04.226 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install python3-rbd 2024-02-14T17:38:04.498 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:02 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:04.611 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:04.661 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:04.662 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-02-14T17:38:04.663 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:04.738 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install python3-rados 2024-02-14T17:38:04.803 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:04 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:38:04.918 INFO:teuthology.orchestra.run.smithi179.stdout:Package python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:04.969 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:38:04.970 INFO:teuthology.orchestra.run.smithi179.stdout:Nothing to do. 2024-02-14T17:38:04.970 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:38:05.024 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-fuse 2024-02-14T17:38:05.307 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:03 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:05.420 INFO:teuthology.orchestra.run.smithi100.stdout:Package python3-rados-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:05.472 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:05.473 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-02-14T17:38:05.473 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:05.547 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install python3-rgw 2024-02-14T17:38:05.596 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:05 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:38:05.763 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:38:05.763 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:38:05.763 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-02-14T17:38:05.764 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:38:05.764 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:38:05.764 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 85 k 2024-02-14T17:38:05.764 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:05.764 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:38:05.764 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:38:05.764 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-02-14T17:38:05.764 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:05.765 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 85 k 2024-02-14T17:38:05.765 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 231 k 2024-02-14T17:38:05.765 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:38:06.096 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-fuse-19.0.0-1430.g8727ce93.el9.x86_64.rpm 257 kB/s | 85 kB 00:00 2024-02-14T17:38:06.097 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:38:06.097 INFO:teuthology.orchestra.run.smithi179.stdout:Total 256 kB/s | 85 kB 00:00 2024-02-14T17:38:06.097 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:38:06.104 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:38:06.104 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:38:06.125 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:04 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:06.161 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:38:06.161 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:38:06.239 INFO:teuthology.orchestra.run.smithi100.stdout:Package python3-rgw-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:06.290 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:06.291 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-02-14T17:38:06.291 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:06.298 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:38:06.364 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install python3-cephfs 2024-02-14T17:38:06.411 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:06.814 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:06.933 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:04 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:07.047 INFO:teuthology.orchestra.run.smithi100.stdout:Package python3-cephfs-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:07.097 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:07.098 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-02-14T17:38:07.098 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:07.137 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:07.137 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:07.137 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:38:07.137 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:38:07.138 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:07.138 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:38:07.172 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install python3-rbd 2024-02-14T17:38:07.331 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-mirror 2024-02-14T17:38:07.741 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:05 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:07.856 INFO:teuthology.orchestra.run.smithi100.stdout:Package python3-rbd-2:19.0.0-1430.g8727ce93.el9.x86_64 is already installed. 2024-02-14T17:38:07.907 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:07.908 INFO:teuthology.orchestra.run.smithi100.stdout:Nothing to do. 2024-02-14T17:38:07.908 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:07.920 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:07 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:38:07.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install rbd-fuse 2024-02-14T17:38:08.093 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:38:08.094 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:38:08.094 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repo Size 2024-02-14T17:38:08.094 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:38:08.094 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:38:08.094 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.1 M 2024-02-14T17:38:08.094 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:08.094 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:38:08.094 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:38:08.094 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-02-14T17:38:08.094 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:08.095 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 3.1 M 2024-02-14T17:38:08.095 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 13 M 2024-02-14T17:38:08.095 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:38:08.550 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:06 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:08.715 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:08.716 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:08.716 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-02-14T17:38:08.716 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:08.716 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:38:08.716 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 85 k 2024-02-14T17:38:08.716 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:08.716 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:38:08.716 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:08.717 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-02-14T17:38:08.717 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:08.717 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 85 k 2024-02-14T17:38:08.717 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 231 k 2024-02-14T17:38:08.717 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:38:08.975 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-mirror-19.0.0-1430.g8727ce93.el9.x86_64.rpm 3.5 MB/s | 3.1 MB 00:00 2024-02-14T17:38:08.975 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:38:08.976 INFO:teuthology.orchestra.run.smithi179.stdout:Total 3.5 MB/s | 3.1 MB 00:00 2024-02-14T17:38:08.976 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:38:08.985 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:38:08.985 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:38:09.058 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:38:09.058 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:38:09.206 INFO:teuthology.orchestra.run.smithi100.stdout:rbd-fuse-19.0.0-1430.g8727ce93.el9.x86_64.rpm 174 kB/s | 85 kB 00:00 2024-02-14T17:38:09.207 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:38:09.207 INFO:teuthology.orchestra.run.smithi100.stdout:Total 174 kB/s | 85 kB 00:00 2024-02-14T17:38:09.208 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:38:09.215 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:38:09.216 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:38:09.270 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:38:09.270 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:38:09.408 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:38:09.479 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:38:09.521 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:09.542 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:09.568 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:09.568 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:38:09.568 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-14T17:38:09.568 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T17:38:09.568 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T17:38:09.568 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:09.906 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:10.247 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:38:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:38:10.247 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:10.247 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:10.491 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install rbd-mirror 2024-02-14T17:38:10.790 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:10.790 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:10.790 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:38:10.790 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:38:10.790 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:10.790 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:38:11.064 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install rbd-nbd 2024-02-14T17:38:11.083 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:09 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:11.251 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repo Size 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 3.1 M 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 3.1 M 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 13 M 2024-02-14T17:38:11.252 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:38:11.654 INFO:teuthology.orchestra.run.smithi179.stdout:Last metadata expiration check: 0:02:11 ago on Wed 14 Feb 2024 05:36:00 PM UTC. 2024-02-14T17:38:11.818 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-02-14T17:38:11.819 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:38:11.819 INFO:teuthology.orchestra.run.smithi179.stdout: Package Architecture Version Repository Size 2024-02-14T17:38:11.819 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:38:11.819 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-02-14T17:38:11.819 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 171 k 2024-02-14T17:38:11.819 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:11.819 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-02-14T17:38:11.820 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-02-14T17:38:11.820 INFO:teuthology.orchestra.run.smithi179.stdout:Install 1 Package 2024-02-14T17:38:11.820 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:11.820 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 171 k 2024-02-14T17:38:11.820 INFO:teuthology.orchestra.run.smithi179.stdout:Installed size: 494 k 2024-02-14T17:38:11.820 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-02-14T17:38:12.162 INFO:teuthology.orchestra.run.smithi179.stdout:rbd-nbd-19.0.0-1430.g8727ce93.el9.x86_64.rpm 500 kB/s | 171 kB 00:00 2024-02-14T17:38:12.162 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:38:12.163 INFO:teuthology.orchestra.run.smithi179.stdout:Total 498 kB/s | 171 kB 00:00 2024-02-14T17:38:12.163 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-02-14T17:38:12.171 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-02-14T17:38:12.171 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-02-14T17:38:12.218 INFO:teuthology.orchestra.run.smithi100.stdout:rbd-mirror-19.0.0-1430.g8727ce93.el9.x86_64.rpm 3.2 MB/s | 3.1 MB 00:00 2024-02-14T17:38:12.218 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:38:12.219 INFO:teuthology.orchestra.run.smithi100.stdout:Total 3.2 MB/s | 3.1 MB 00:00 2024-02-14T17:38:12.219 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:38:12.227 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-02-14T17:38:12.227 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-02-14T17:38:12.228 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:38:12.229 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:38:12.302 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:38:12.303 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:38:12.381 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-02-14T17:38:12.485 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:12.724 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:38:12.785 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:12.810 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:12.810 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-14T17:38:12.810 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-14T17:38:12.810 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T17:38:12.811 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-14T17:38:12.811 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:12.907 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:13.196 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:13.196 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:13.196 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-02-14T17:38:13.196 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:38:13.196 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:38:13.196 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-02-14T17:38:13.416 DEBUG:teuthology.parallel:result is None 2024-02-14T17:38:13.984 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:13.984 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:13.984 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:38:13.985 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:38:13.985 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:13.985 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:14.220 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install rbd-nbd 2024-02-14T17:38:14.808 INFO:teuthology.orchestra.run.smithi100.stdout:Last metadata expiration check: 0:02:12 ago on Wed 14 Feb 2024 05:36:02 PM UTC. 2024-02-14T17:38:14.972 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-02-14T17:38:14.973 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:14.973 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-02-14T17:38:14.973 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:14.973 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-02-14T17:38:14.973 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.0.0-1430.g8727ce93.el9 ceph 171 k 2024-02-14T17:38:14.974 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:14.974 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-02-14T17:38:14.974 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-02-14T17:38:14.974 INFO:teuthology.orchestra.run.smithi100.stdout:Install 1 Package 2024-02-14T17:38:14.974 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:14.974 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 171 k 2024-02-14T17:38:14.974 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 494 k 2024-02-14T17:38:14.974 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-02-14T17:38:15.347 INFO:teuthology.orchestra.run.smithi100.stdout:rbd-nbd-19.0.0-1430.g8727ce93.el9.x86_64.rpm 458 kB/s | 171 kB 00:00 2024-02-14T17:38:15.348 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:38:15.348 INFO:teuthology.orchestra.run.smithi100.stdout:Total 456 kB/s | 171 kB 00:00 2024-02-14T17:38:15.348 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-02-14T17:38:15.355 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-02-14T17:38:15.356 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-02-14T17:38:15.411 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-02-14T17:38:15.411 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-02-14T17:38:15.556 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-02-14T17:38:15.678 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:16.141 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 1/1 2024-02-14T17:38:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:16.476 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-02-14T17:38:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.0.0-1430.g8727ce93.el9.x86_64 2024-02-14T17:38:16.476 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:38:16.476 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-02-14T17:38:16.717 DEBUG:teuthology.parallel:result is None 2024-02-14T17:38:16.718 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:38:16.862 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-14T17:38:16.883 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-1430.g8727ce93.el9 2024-02-14T17:38:16.883 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1430.g8727ce93.el9 2024-02-14T17:38:16.883 INFO:teuthology.task.install:The correct ceph version 19.0.0-1430.g8727ce93 is installed. 2024-02-14T17:38:16.885 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:38:17.019 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-14T17:38:17.044 INFO:teuthology.orchestra.run.smithi179.stdout:19.0.0-1430.g8727ce93.el9 2024-02-14T17:38:17.044 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1430.g8727ce93.el9 2024-02-14T17:38:17.044 INFO:teuthology.task.install:The correct ceph version 19.0.0-1430.g8727ce93 is installed. 2024-02-14T17:38:17.046 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-14T17:38:17.046 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:38:17.046 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-14T17:38:17.074 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:38:17.074 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-14T17:38:17.116 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-14T17:38:17.116 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:38:17.116 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-02-14T17:38:17.143 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-14T17:38:17.211 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:38:17.211 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-02-14T17:38:17.241 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-14T17:38:17.310 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-14T17:38:17.310 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:38:17.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-14T17:38:17.337 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-14T17:38:17.403 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:38:17.403 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-14T17:38:17.432 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-14T17:38:17.500 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-14T17:38:17.501 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:38:17.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-02-14T17:38:17.527 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-14T17:38:17.594 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:38:17.594 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-02-14T17:38:17.621 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-14T17:38:17.689 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-14T17:38:17.784 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1', 'cephadm_mode': 'root'} 2024-02-14T17:38:17.784 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:38:17.785 INFO:tasks.cephadm:Cluster fsid is d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:38:17.785 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-14T17:38:17.785 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.100', 'mon.c': '[v2:172.21.15.100:3301,v1:172.21.15.100:6790]', 'mon.b': '172.21.15.179'} 2024-02-14T17:38:17.785 INFO:tasks.cephadm:First mon is mon.a on smithi100 2024-02-14T17:38:17.785 INFO:tasks.cephadm:First mgr is y 2024-02-14T17:38:17.785 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-14T17:38:17.786 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-02-14T17:38:17.813 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname $(hostname -s) 2024-02-14T17:38:17.841 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-14T17:38:17.841 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:38:18.057 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8727ce939fc044a98f215e29696bb1433f5758d1', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76734/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1430-g8727ce93', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1430.g8727ce93'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-14 13:20:25.883975', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-testing-2024-02-14-1033', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-14T17:38:18.270 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-guits-testing-2024-02-14-1033, sha1 8727ce939fc044a98f215e29696bb1433f5758d1 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:38:18.271 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/x86_64/flavors/default/cephadm 2024-02-14T17:38:18.271 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-14T17:38:19.038 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 767024 Feb 14 17:38 /home/ubuntu/cephtest/cephadm 2024-02-14T17:38:19.038 DEBUG:teuthology.orchestra.run.smithi179:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-guits-testing-2024-02-14-1033/8727ce939fc044a98f215e29696bb1433f5758d1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-14T17:38:19.763 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r--. 1 ubuntu ubuntu 767024 Feb 14 17:38 /home/ubuntu/cephtest/cephadm 2024-02-14T17:38:19.764 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-14T17:38:19.782 DEBUG:teuthology.orchestra.run.smithi179:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-14T17:38:19.812 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 on all hosts... 2024-02-14T17:38:19.812 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 pull 2024-02-14T17:38:19.825 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 pull 2024-02-14T17:38:20.697 INFO:teuthology.orchestra.run.smithi179.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:38:20.766 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:38:56.388 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-02-14T17:38:56.388 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:38:56.388 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86", 2024-02-14T17:38:56.388 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-02-14T17:38:56.388 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b" 2024-02-14T17:38:56.388 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-02-14T17:38:56.388 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-02-14T17:39:06.289 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-02-14T17:39:06.289 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev)", 2024-02-14T17:39:06.289 INFO:teuthology.orchestra.run.smithi179.stdout: "image_id": "170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86", 2024-02-14T17:39:06.289 INFO:teuthology.orchestra.run.smithi179.stdout: "repo_digests": [ 2024-02-14T17:39:06.290 INFO:teuthology.orchestra.run.smithi179.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b" 2024-02-14T17:39:06.290 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-02-14T17:39:06.290 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-02-14T17:39:06.313 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-02-14T17:39:06.349 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph 2024-02-14T17:39:06.379 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-02-14T17:39:06.418 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /etc/ceph 2024-02-14T17:39:06.446 INFO:tasks.cephadm:Writing seed config... 2024-02-14T17:39:06.448 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-14T17:39:06.448 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-14T17:39:06.448 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-14T17:39:06.448 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-02-14T17:39:06.448 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-14T17:39:06.448 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-14T17:39:06.448 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-14T17:39:06.448 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-14T17:39:06.448 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-14T17:39:06.449 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-14T17:39:06.449 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-14T17:39:06.449 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-02-14T17:39:06.449 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-02-14T17:39:06.449 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-14T17:39:06.449 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:39:06.450 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-14T17:39:06.475 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d69a9998-cb5f-11ee-95b9-87774f69a715 mon election default strategy = 3 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-02-14T17:39:06.475 DEBUG:teuthology.orchestra.run.smithi100:mon.a> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.a.service 2024-02-14T17:39:06.518 DEBUG:teuthology.orchestra.run.smithi100:mgr.y> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.y.service 2024-02-14T17:39:06.562 INFO:tasks.cephadm:Bootstrapping... 2024-02-14T17:39:06.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 -v bootstrap --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.100 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:39:06.759 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-02-14T17:39:06.759 INFO:teuthology.orchestra.run.smithi100.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1', '-v', 'bootstrap', '--fsid', 'd69a9998-cb5f-11ee-95b9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.100', '--skip-admin-label'] 2024-02-14T17:39:06.784 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T17:39:06.784 INFO:teuthology.orchestra.run.smithi100.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-14T17:39:06.784 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying podman|docker is present... 2024-02-14T17:39:06.808 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T17:39:06.808 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying lvm2 is present... 2024-02-14T17:39:06.808 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying time synchronization is in place... 2024-02-14T17:39:06.815 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-14T17:39:06.815 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-14T17:39:06.821 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-14T17:39:06.821 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-02-14T17:39:06.829 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-02-14T17:39:06.835 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-02-14T17:39:06.836 INFO:teuthology.orchestra.run.smithi100.stdout:Unit chronyd.service is enabled and running 2024-02-14T17:39:06.836 INFO:teuthology.orchestra.run.smithi100.stdout:Repeating the final host check... 2024-02-14T17:39:06.862 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-14T17:39:06.862 INFO:teuthology.orchestra.run.smithi100.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-14T17:39:06.862 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl is present 2024-02-14T17:39:06.862 INFO:teuthology.orchestra.run.smithi100.stdout:lvcreate is present 2024-02-14T17:39:06.868 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-14T17:39:06.868 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-14T17:39:06.875 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-14T17:39:06.875 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-02-14T17:39:06.881 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout enabled 2024-02-14T17:39:06.888 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout active 2024-02-14T17:39:06.888 INFO:teuthology.orchestra.run.smithi100.stdout:Unit chronyd.service is enabled and running 2024-02-14T17:39:06.888 INFO:teuthology.orchestra.run.smithi100.stdout:Host looks OK 2024-02-14T17:39:06.888 INFO:teuthology.orchestra.run.smithi100.stdout:Cluster fsid: d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:06.888 INFO:teuthology.orchestra.run.smithi100.stdout:Acquiring lock 139876375478224 on /run/cephadm/d69a9998-cb5f-11ee-95b9-87774f69a715.lock 2024-02-14T17:39:06.888 INFO:teuthology.orchestra.run.smithi100.stdout:Lock 139876375478224 acquired on /run/cephadm/d69a9998-cb5f-11ee-95b9-87774f69a715.lock 2024-02-14T17:39:06.889 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 3300 ... 2024-02-14T17:39:06.889 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying IP 172.21.15.100 port 6789 ... 2024-02-14T17:39:06.889 INFO:teuthology.orchestra.run.smithi100.stdout:Base mon IP(s) is [172.21.15.100:3300, 172.21.15.100:6789], mon addrv is [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-02-14T17:39:06.892 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.100 metric 100 2024-02-14T17:39:06.892 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.100 metric 100 2024-02-14T17:39:06.895 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-14T17:39:06.895 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-14T17:39:06.895 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1788sec hoplimit 64 pref medium 2024-02-14T17:39:06.897 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-14T17:39:06.897 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-14T17:39:06.897 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-14T17:39:06.897 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-14T17:39:06.897 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d733/64 scope link 2024-02-14T17:39:06.897 INFO:teuthology.orchestra.run.smithi100.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-14T17:39:06.898 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-02-14T17:39:06.898 INFO:teuthology.orchestra.run.smithi100.stdout:Mon IP `172.21.15.100` is in CIDR network `172.21.0.0/20` 2024-02-14T17:39:06.898 INFO:teuthology.orchestra.run.smithi100.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-14T17:39:06.898 INFO:teuthology.orchestra.run.smithi100.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-14T17:39:06.899 INFO:teuthology.orchestra.run.smithi100.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:39:10.337 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stdout 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 2024-02-14T17:39:10.337 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1... 2024-02-14T17:39:10.337 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Getting image source signatures 2024-02-14T17:39:10.337 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying blob sha256:85deddcd78c19bc18770d0f2ed0eee83c11fab2d5e1d2ebb809a77baeec400f6 2024-02-14T17:39:10.337 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-14T17:39:10.337 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Copying config sha256:170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 2024-02-14T17:39:10.337 INFO:teuthology.orchestra.run.smithi100.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-14T17:39:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:ceph: stdout ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev) 2024-02-14T17:39:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph version: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev) 2024-02-14T17:39:14.550 INFO:teuthology.orchestra.run.smithi100.stdout:Extracting ceph user uid/gid from container image... 2024-02-14T17:39:15.553 INFO:teuthology.orchestra.run.smithi100.stdout:stat: stdout 167 167 2024-02-14T17:39:15.554 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial keys... 2024-02-14T17:39:16.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQDD+sxlr4qLNhAAyeo4ksqQiSOvU8VGfoG1NA== 2024-02-14T17:39:17.548 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQDE+sxlu2lUMRAAVqRPjmgxti08rgrLcb4cUQ== 2024-02-14T17:39:20.320 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph-authtool: stdout AQDG+sxlkUvcIBAAx9pH7rkhMzmzXPVb1eZKiA== 2024-02-14T17:39:20.321 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial monmap... 2024-02-14T17:39:21.417 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-14T17:39:21.418 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-14T17:39:21.418 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:21.418 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-14T17:39:21.418 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool for a [v2:172.21.15.100:3300,v1:172.21.15.100:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-14T17:39:21.418 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = quincy 2024-02-14T17:39:21.418 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: set fsid to d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:21.418 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-14T17:39:21.418 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:39:21.418 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mon... 2024-02-14T17:39:22.431 INFO:teuthology.orchestra.run.smithi100.stdout:create mon.a on 2024-02-14T17:39:22.638 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-14T17:39:22.781 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-14T17:39:22.950 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d69a9998-cb5f-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-d69a9998-cb5f-11ee-95b9-87774f69a715.target. 2024-02-14T17:39:22.950 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d69a9998-cb5f-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-d69a9998-cb5f-11ee-95b9-87774f69a715.target. 2024-02-14T17:39:23.184 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.a 2024-02-14T17:39:23.185 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.a.service: Unit ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.a.service not loaded. 2024-02-14T17:39:23.342 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d69a9998-cb5f-11ee-95b9-87774f69a715.target.wants/ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@.service. 2024-02-14T17:39:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:23 smithi100 podman[29612]: 2024-02-14 17:39:23.588162035 +0000 UTC m=+0.115789158 container create 881e33e79ae0a446add8d913428f0cc7b99d9ab015f6f5db408296142ad2bc3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T17:39:23.863 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:39:23.863 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-02-14T17:39:23.870 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:39:23.870 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-02-14T17:39:23.870 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-02-14T17:39:23.870 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-02-14T17:39:23.870 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon to start... 2024-02-14T17:39:23.870 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mon... 2024-02-14T17:39:24.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:23 smithi100 podman[29612]: 2024-02-14 17:39:23.782114307 +0000 UTC m=+0.309741433 container init 881e33e79ae0a446add8d913428f0cc7b99d9ab015f6f5db408296142ad2bc3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-14T17:39:24.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:23 smithi100 podman[29612]: 2024-02-14 17:39:23.787368463 +0000 UTC m=+0.314995587 container start 881e33e79ae0a446add8d913428f0cc7b99d9ab015f6f5db408296142ad2bc3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T17:39:24.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:23 smithi100 bash[29612]: 881e33e79ae0a446add8d913428f0cc7b99d9ab015f6f5db408296142ad2bc3a 2024-02-14T17:39:24.112 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:23 smithi100 systemd[1]: Started Ceph mon.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:39:24.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:23 smithi100 ceph-mon[29649]: mkfs d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:24.113 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:23 smithi100 ceph-mon[29649]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-14T17:39:25.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:24 smithi100 ceph-mon[29649]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-14T17:39:25.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:24 smithi100 ceph-mon[29649]: monmap e1: 1 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:39:25.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:24 smithi100 ceph-mon[29649]: fsmap 2024-02-14T17:39:25.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:24 smithi100 ceph-mon[29649]: osdmap e1: 0 total, 0 up, 0 in 2024-02-14T17:39:25.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:24 smithi100 ceph-mon[29649]: mgrmap e1: no daemons active 2024-02-14T17:39:25.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:24 smithi100 ceph-mon[29649]: from='client.? 172.21.15.100:0/3040131511' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-14T17:39:25.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout cluster: 2024-02-14T17:39:25.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout id: d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:25.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-14T17:39:25.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:25.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout services: 2024-02-14T17:39:25.140 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.691242s) 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout data: 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout pgs: 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.469+0000 7f4f18166700 1 Processor -- start 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.469+0000 7f4f18166700 1 -- start start 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.469+0000 7f4f18166700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10104d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.469+0000 7f4f18166700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f10105320 con 0x7f4f10104930 2024-02-14T17:39:25.141 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.469+0000 7f4f15f02700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10104d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.470+0000 7f4f15f02700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10104d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43254/0 (socket says 172.21.15.100:43254) 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.470+0000 7f4f15f02700 1 -- 172.21.15.100:0/608516848 learned_addr learned my addr 172.21.15.100:0/608516848 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.470+0000 7f4f15f02700 1 -- 172.21.15.100:0/608516848 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f10105b30 con 0x7f4f10104930 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.470+0000 7f4f15f02700 1 --2- 172.21.15.100:0/608516848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10104d50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0098a0 tx=0x7f4f0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe4ab579bcab387 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.470+0000 7f4f14f00700 1 -- 172.21.15.100:0/608516848 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f0c004160 con 0x7f4f10104930 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.470+0000 7f4f14f00700 1 -- 172.21.15.100:0/608516848 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c0042c0 con 0x7f4f10104930 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.470+0000 7f4f14f00700 1 -- 172.21.15.100:0/608516848 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f0c0045a0 con 0x7f4f10104930 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.471+0000 7f4f18166700 1 -- 172.21.15.100:0/608516848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 msgr2=0x7f4f10104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.471+0000 7f4f18166700 1 --2- 172.21.15.100:0/608516848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10104d50 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0098a0 tx=0x7f4f0c009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.471+0000 7f4f18166700 1 -- 172.21.15.100:0/608516848 shutdown_connections 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.471+0000 7f4f18166700 1 --2- 172.21.15.100:0/608516848 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10104d50 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.471+0000 7f4f18166700 1 -- 172.21.15.100:0/608516848 >> 172.21.15.100:0/608516848 conn(0x7f4f100ffef0 msgr2=0x7f4f10102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.471+0000 7f4f18166700 1 -- 172.21.15.100:0/608516848 shutdown_connections 2024-02-14T17:39:25.142 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.471+0000 7f4f18166700 1 -- 172.21.15.100:0/608516848 wait complete. 2024-02-14T17:39:25.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.471+0000 7f4f18166700 1 Processor -- start 2024-02-14T17:39:25.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.471+0000 7f4f18166700 1 -- start start 2024-02-14T17:39:25.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f18166700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:25.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f18166700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f10198320 con 0x7f4f10104930 2024-02-14T17:39:25.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f15f02700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:25.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f15f02700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43258/0 (socket says 172.21.15.100:43258) 2024-02-14T17:39:25.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f15f02700 1 -- 172.21.15.100:0/3040131511 learned_addr learned my addr 172.21.15.100:0/3040131511 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:25.143 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f15f02700 1 -- 172.21.15.100:0/3040131511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c009550 con 0x7f4f10104930 2024-02-14T17:39:25.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f15f02700 1 --2- 172.21.15.100:0/3040131511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10197de0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c004000 tx=0x7f4f0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:25.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f06ffd700 1 -- 172.21.15.100:0/3040131511 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f0c011630 con 0x7f4f10104930 2024-02-14T17:39:25.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f06ffd700 1 -- 172.21.15.100:0/3040131511 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c011790 con 0x7f4f10104930 2024-02-14T17:39:25.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f18166700 1 -- 172.21.15.100:0/3040131511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f10198520 con 0x7f4f10104930 2024-02-14T17:39:25.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f06ffd700 1 -- 172.21.15.100:0/3040131511 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f0c011a30 con 0x7f4f10104930 2024-02-14T17:39:25.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f18166700 1 -- 172.21.15.100:0/3040131511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f101989c0 con 0x7f4f10104930 2024-02-14T17:39:25.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f06ffd700 1 -- 172.21.15.100:0/3040131511 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4f0c022070 con 0x7f4f10104930 2024-02-14T17:39:25.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.472+0000 7f4f06ffd700 1 -- 172.21.15.100:0/3040131511 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4f0c01c450 con 0x7f4f10104930 2024-02-14T17:39:25.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.473+0000 7f4f18166700 1 -- 172.21.15.100:0/3040131511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f10191ab0 con 0x7f4f10104930 2024-02-14T17:39:25.144 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.475+0000 7f4f06ffd700 1 -- 172.21.15.100:0/3040131511 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4f0c011b90 con 0x7f4f10104930 2024-02-14T17:39:25.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.544+0000 7f4f18166700 1 -- 172.21.15.100:0/3040131511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f4f101060d0 con 0x7f4f10104930 2024-02-14T17:39:25.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.544+0000 7f4f06ffd700 1 -- 172.21.15.100:0/3040131511 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f4f0c011d30 con 0x7f4f10104930 2024-02-14T17:39:25.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.545+0000 7f4f18166700 1 -- 172.21.15.100:0/3040131511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 msgr2=0x7f4f10197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:25.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.545+0000 7f4f18166700 1 --2- 172.21.15.100:0/3040131511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10197de0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c004000 tx=0x7f4f0c004750 comp rx=0 tx=0).stop 2024-02-14T17:39:25.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.545+0000 7f4f18166700 1 -- 172.21.15.100:0/3040131511 shutdown_connections 2024-02-14T17:39:25.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.545+0000 7f4f18166700 1 --2- 172.21.15.100:0/3040131511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f10104930 0x7f4f10197de0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:25.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.545+0000 7f4f18166700 1 -- 172.21.15.100:0/3040131511 >> 172.21.15.100:0/3040131511 conn(0x7f4f100ffef0 msgr2=0x7f4f1018ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:25.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.545+0000 7f4f18166700 1 -- 172.21.15.100:0/3040131511 shutdown_connections 2024-02-14T17:39:25.145 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:24.545+0000 7f4f18166700 1 -- 172.21.15.100:0/3040131511 wait complete. 2024-02-14T17:39:25.145 INFO:teuthology.orchestra.run.smithi100.stdout:mon is available 2024-02-14T17:39:25.146 INFO:teuthology.orchestra.run.smithi100.stdout:Assimilating anything we can from ceph.conf... 2024-02-14T17:39:26.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:26 smithi100 ceph-mon[29649]: from='client.? 172.21.15.100:0/2497509439' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-14T17:39:26.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:26 smithi100 ceph-mon[29649]: from='client.? 172.21.15.100:0/2497509439' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-14T17:39:26.480 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:26.480 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-02-14T17:39:26.480 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:26.480 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.100:3300,v1:172.21.15.100:6789] 2024-02-14T17:39:26.480 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-14T17:39:26.480 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-14T17:39:26.480 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-14T17:39:26.480 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-14T17:39:26.480 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.936+0000 7f18546ac700 1 Processor -- start 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.936+0000 7f18546ac700 1 -- start start 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.937+0000 7f18546ac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c106300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.937+0000 7f18546ac700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c1068d0 con 0x7f184c105ee0 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.937+0000 7f1852448700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c106300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.937+0000 7f1852448700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c106300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43272/0 (socket says 172.21.15.100:43272) 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.937+0000 7f1852448700 1 -- 172.21.15.100:0/2159091753 learned_addr learned my addr 172.21.15.100:0/2159091753 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:26.481 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.937+0000 7f1852448700 1 -- 172.21.15.100:0/2159091753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c104af0 con 0x7f184c105ee0 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.937+0000 7f1852448700 1 --2- 172.21.15.100:0/2159091753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c106300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f18480098a0 tx=0x7f1848009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb06427a43d6b3f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.938+0000 7f1851446700 1 -- 172.21.15.100:0/2159091753 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1848004160 con 0x7f184c105ee0 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.938+0000 7f1851446700 1 -- 172.21.15.100:0/2159091753 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18480042c0 con 0x7f184c105ee0 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.938+0000 7f1851446700 1 -- 172.21.15.100:0/2159091753 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18480045a0 con 0x7f184c105ee0 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.938+0000 7f18546ac700 1 -- 172.21.15.100:0/2159091753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 msgr2=0x7f184c106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.938+0000 7f18546ac700 1 --2- 172.21.15.100:0/2159091753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c106300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f18480098a0 tx=0x7f1848009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.938+0000 7f18546ac700 1 -- 172.21.15.100:0/2159091753 shutdown_connections 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.938+0000 7f18546ac700 1 --2- 172.21.15.100:0/2159091753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c106300 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.938+0000 7f18546ac700 1 -- 172.21.15.100:0/2159091753 >> 172.21.15.100:0/2159091753 conn(0x7f184c0ff640 msgr2=0x7f184c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.938+0000 7f18546ac700 1 -- 172.21.15.100:0/2159091753 shutdown_connections 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.938+0000 7f18546ac700 1 -- 172.21.15.100:0/2159091753 wait complete. 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.939+0000 7f18546ac700 1 Processor -- start 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.939+0000 7f18546ac700 1 -- start start 2024-02-14T17:39:26.482 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.939+0000 7f18546ac700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.939+0000 7f18546ac700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c1994a0 con 0x7f184c105ee0 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.939+0000 7f1852448700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.939+0000 7f1852448700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43282/0 (socket says 172.21.15.100:43282) 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.939+0000 7f1852448700 1 -- 172.21.15.100:0/2497509439 learned_addr learned my addr 172.21.15.100:0/2497509439 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.939+0000 7f1852448700 1 -- 172.21.15.100:0/2497509439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1848009550 con 0x7f184c105ee0 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.940+0000 7f1852448700 1 --2- 172.21.15.100:0/2497509439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c198f60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1848004000 tx=0x7f1848004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.940+0000 7f18437fe700 1 -- 172.21.15.100:0/2497509439 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1848011630 con 0x7f184c105ee0 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.940+0000 7f18437fe700 1 -- 172.21.15.100:0/2497509439 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1848011790 con 0x7f184c105ee0 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.940+0000 7f18546ac700 1 -- 172.21.15.100:0/2497509439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184c1996a0 con 0x7f184c105ee0 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.940+0000 7f18437fe700 1 -- 172.21.15.100:0/2497509439 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1848011a30 con 0x7f184c105ee0 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.940+0000 7f18546ac700 1 -- 172.21.15.100:0/2497509439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c199b40 con 0x7f184c105ee0 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.940+0000 7f18437fe700 1 -- 172.21.15.100:0/2497509439 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1848021070 con 0x7f184c105ee0 2024-02-14T17:39:26.483 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.940+0000 7f18437fe700 1 -- 172.21.15.100:0/2497509439 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f184801b810 con 0x7f184c105ee0 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.941+0000 7f18546ac700 1 -- 172.21.15.100:0/2497509439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184c192b80 con 0x7f184c105ee0 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:25.943+0000 7f18437fe700 1 -- 172.21.15.100:0/2497509439 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f184801ba70 con 0x7f184c105ee0 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.013+0000 7f18546ac700 1 -- 172.21.15.100:0/2497509439 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f184c104e60 con 0x7f184c105ee0 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.016+0000 7f18437fe700 1 -- 172.21.15.100:0/2497509439 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+340 (secure 0 0 0) 0x7f184801bcc0 con 0x7f184c105ee0 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.017+0000 7f18437fe700 1 -- 172.21.15.100:0/2497509439 <== mon.0 v2:172.21.15.100:3300/0 8 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f184801be60 con 0x7f184c105ee0 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.018+0000 7f18546ac700 1 -- 172.21.15.100:0/2497509439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 msgr2=0x7f184c198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.018+0000 7f18546ac700 1 --2- 172.21.15.100:0/2497509439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c198f60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1848004000 tx=0x7f1848004750 comp rx=0 tx=0).stop 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.018+0000 7f18546ac700 1 -- 172.21.15.100:0/2497509439 shutdown_connections 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.018+0000 7f18546ac700 1 --2- 172.21.15.100:0/2497509439 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f184c105ee0 0x7f184c198f60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.018+0000 7f18546ac700 1 -- 172.21.15.100:0/2497509439 >> 172.21.15.100:0/2497509439 conn(0x7f184c0ff640 msgr2=0x7f184c18fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.018+0000 7f18546ac700 1 -- 172.21.15.100:0/2497509439 shutdown_connections 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.018+0000 7f18546ac700 1 -- 172.21.15.100:0/2497509439 wait complete. 2024-02-14T17:39:26.484 INFO:teuthology.orchestra.run.smithi100.stdout:Generating new minimal ceph.conf... 2024-02-14T17:39:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:27 smithi100 ceph-mon[29649]: from='client.? 172.21.15.100:0/1651025877' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:39:27.584 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.914+0000 7f91733ae700 1 Processor -- start 2024-02-14T17:39:27.584 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.914+0000 7f91733ae700 1 -- start start 2024-02-14T17:39:27.584 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.914+0000 7f91733ae700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c107550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:27.584 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.914+0000 7f91733ae700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c107b20 con 0x7f916c107130 2024-02-14T17:39:27.584 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.914+0000 7f917114a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:27.584 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.914+0000 7f917114a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c107550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43286/0 (socket says 172.21.15.100:43286) 2024-02-14T17:39:27.584 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.914+0000 7f917114a700 1 -- 172.21.15.100:0/3243338951 learned_addr learned my addr 172.21.15.100:0/3243338951 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:27.584 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.914+0000 7f917114a700 1 -- 172.21.15.100:0/3243338951 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c105670 con 0x7f916c107130 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.914+0000 7f917114a700 1 --2- 172.21.15.100:0/3243338951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c107550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f91640098a0 tx=0x7f9164009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca9c84dca9753cb server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.915+0000 7f916bfff700 1 -- 172.21.15.100:0/3243338951 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9164004160 con 0x7f916c107130 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.915+0000 7f916bfff700 1 -- 172.21.15.100:0/3243338951 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f91640042c0 con 0x7f916c107130 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.915+0000 7f916bfff700 1 -- 172.21.15.100:0/3243338951 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91640118a0 con 0x7f916c107130 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.915+0000 7f91733ae700 1 -- 172.21.15.100:0/3243338951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 msgr2=0x7f916c107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.915+0000 7f91733ae700 1 --2- 172.21.15.100:0/3243338951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c107550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f91640098a0 tx=0x7f9164009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.915+0000 7f91733ae700 1 -- 172.21.15.100:0/3243338951 shutdown_connections 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.915+0000 7f91733ae700 1 --2- 172.21.15.100:0/3243338951 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c107550 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.915+0000 7f91733ae700 1 -- 172.21.15.100:0/3243338951 >> 172.21.15.100:0/3243338951 conn(0x7f916c101030 msgr2=0x7f916c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.915+0000 7f91733ae700 1 -- 172.21.15.100:0/3243338951 shutdown_connections 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.915+0000 7f91733ae700 1 -- 172.21.15.100:0/3243338951 wait complete. 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.916+0000 7f91733ae700 1 Processor -- start 2024-02-14T17:39:27.585 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.916+0000 7f91733ae700 1 -- start start 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.916+0000 7f91733ae700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.916+0000 7f91733ae700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c199c80 con 0x7f916c107130 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.916+0000 7f917114a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.916+0000 7f917114a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43290/0 (socket says 172.21.15.100:43290) 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.916+0000 7f917114a700 1 -- 172.21.15.100:0/1651025877 learned_addr learned my addr 172.21.15.100:0/1651025877 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.916+0000 7f917114a700 1 -- 172.21.15.100:0/1651025877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9164009550 con 0x7f916c107130 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.916+0000 7f917114a700 1 --2- 172.21.15.100:0/1651025877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c199740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9164003da0 tx=0x7f9164011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.917+0000 7f916a7fc700 1 -- 172.21.15.100:0/1651025877 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9164011930 con 0x7f916c107130 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.917+0000 7f916a7fc700 1 -- 172.21.15.100:0/1651025877 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f9164003b00 con 0x7f916c107130 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.917+0000 7f91733ae700 1 -- 172.21.15.100:0/1651025877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c199e80 con 0x7f916c107130 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.917+0000 7f916a7fc700 1 -- 172.21.15.100:0/1651025877 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f916401ad50 con 0x7f916c107130 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.917+0000 7f91733ae700 1 -- 172.21.15.100:0/1651025877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c19a320 con 0x7f916c107130 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.917+0000 7f916a7fc700 1 -- 172.21.15.100:0/1651025877 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f916402d7d0 con 0x7f916c107130 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.917+0000 7f916a7fc700 1 -- 172.21.15.100:0/1651025877 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9164019700 con 0x7f916c107130 2024-02-14T17:39:27.586 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.918+0000 7f91733ae700 1 -- 172.21.15.100:0/1651025877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c0632a0 con 0x7f916c107130 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.920+0000 7f916a7fc700 1 -- 172.21.15.100:0/1651025877 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f916401a630 con 0x7f916c107130 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.986+0000 7f91733ae700 1 -- 172.21.15.100:0/1651025877 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f916c19cc00 con 0x7f916c107130 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.987+0000 7f916a7fc700 1 -- 172.21.15.100:0/1651025877 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f916402dc10 con 0x7f916c107130 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.988+0000 7f91733ae700 1 -- 172.21.15.100:0/1651025877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 msgr2=0x7f916c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.988+0000 7f91733ae700 1 --2- 172.21.15.100:0/1651025877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c199740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9164003da0 tx=0x7f9164011e90 comp rx=0 tx=0).stop 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.988+0000 7f91733ae700 1 -- 172.21.15.100:0/1651025877 shutdown_connections 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.988+0000 7f91733ae700 1 --2- 172.21.15.100:0/1651025877 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f916c107130 0x7f916c199740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.988+0000 7f91733ae700 1 -- 172.21.15.100:0/1651025877 >> 172.21.15.100:0/1651025877 conn(0x7f916c101030 msgr2=0x7f916c101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.988+0000 7f91733ae700 1 -- 172.21.15.100:0/1651025877 shutdown_connections 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:26.988+0000 7f91733ae700 1 -- 172.21.15.100:0/1651025877 wait complete. 2024-02-14T17:39:27.587 INFO:teuthology.orchestra.run.smithi100.stdout:Restarting the monitor... 2024-02-14T17:39:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:27 smithi100 systemd[1]: Stopping Ceph mon.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:39:28.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:27 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a[29626]: 2024-02-14T17:39:27.839+0000 7eff59965700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T17:39:28.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:27 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a[29626]: 2024-02-14T17:39:27.839+0000 7eff59965700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-14T17:39:28.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:27 smithi100 podman[29980]: 2024-02-14 17:39:27.942293734 +0000 UTC m=+0.206039601 container died 881e33e79ae0a446add8d913428f0cc7b99d9ab015f6f5db408296142ad2bc3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-14T17:39:28.580 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:28 smithi100 podman[29980]: 2024-02-14 17:39:28.328377963 +0000 UTC m=+0.592123830 container cleanup 881e33e79ae0a446add8d913428f0cc7b99d9ab015f6f5db408296142ad2bc3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-14T17:39:28.580 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:28 smithi100 bash[29980]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a 2024-02-14T17:39:28.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:28 smithi100 podman[30000]: 2024-02-14 17:39:28.579090838 +0000 UTC m=+0.629108596 container remove 881e33e79ae0a446add8d913428f0cc7b99d9ab015f6f5db408296142ad2bc3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux ) 2024-02-14T17:39:28.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:28 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-14T17:39:28.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:28 smithi100 systemd[1]: Stopped Ceph mon.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:39:28.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:28 smithi100 systemd[1]: Starting Ceph mon.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:28 smithi100 podman[30100]: 2024-02-14 17:39:28.863286115 +0000 UTC m=+0.017304715 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:28 smithi100 podman[30100]: 2024-02-14 17:39:28.963173353 +0000 UTC m=+0.117191950 container create 79efe7e1a03d9550542f6899f0b3c280f92592f974c853059781ba8a2c9e0883 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 podman[30100]: 2024-02-14 17:39:29.140410475 +0000 UTC m=+0.294429074 container init 79efe7e1a03d9550542f6899f0b3c280f92592f974c853059781ba8a2c9e0883 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 podman[30100]: 2024-02-14 17:39:29.145817955 +0000 UTC m=+0.299836553 container start 79efe7e1a03d9550542f6899f0b3c280f92592f974c853059781ba8a2c9e0883 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS) 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev), process ceph-mon, pid 7 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: pidfile_write: ignore empty --pid-file 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: load: jerasure load: lrc 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: RocksDB version: 7.9.2 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Git sha 0 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Compile date 2024-02-14 11:59:27 2024-02-14T17:39:29.225 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: DB SUMMARY 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: DB Session ID: W6NRKTVP69VKEWBGJOWE 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: CURRENT file: CURRENT 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 86845 ; 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.error_if_exists: 0 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.create_if_missing: 0 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.paranoid_checks: 1 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.env: 0x557c72022ae0 2024-02-14T17:39:29.226 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.info_log: 0x557c731a58c0 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.statistics: (nil) 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.use_fsync: 0 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_log_file_size: 0 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.allow_fallocate: 1 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.use_direct_reads: 0 2024-02-14T17:39:29.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.db_log_dir: 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.wal_dir: 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.write_buffer_manager: 0x557c73179400 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T17:39:29.228 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T17:39:29.229 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T17:39:29.229 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T17:39:29.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T17:39:29.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T17:39:29.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T17:39:29.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.unordered_write: 0 2024-02-14T17:39:29.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T17:39:29.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T17:39:29.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T17:39:29.230 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.row_cache: None 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.wal_filter: None 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.two_write_queues: 0 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.wal_compression: 0 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.atomic_flush: 0 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.log_readahead_size: 0 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T17:39:29.231 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_background_jobs: 2 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_background_compactions: -1 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_subcompactions: 1 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T17:39:29.232 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_open_files: -1 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_background_flushes: -1 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Compression algorithms supported: 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: kZSTD supported: 0 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: kXpressCompression supported: 0 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: kZlibCompression supported: 1 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: kSnappyCompression supported: 1 2024-02-14T17:39:29.233 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: kLZ4Compression supported: 1 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: kBZip2Compression supported: 0 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.merge_operator: 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_filter: None 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T17:39:29.234 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557c731a59c0) 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: cache_index_and_filter_blocks: 1 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: pin_top_level_index_and_filter: 1 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_type: 0 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: data_block_index_type: 0 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_shortening: 1 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: checksum: 4 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: no_block_cache: 0 2024-02-14T17:39:29.235 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache: 0x557c731a9090 2024-02-14T17:39:29.236 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_name: BinnedLRUCache 2024-02-14T17:39:29.236 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_options: 2024-02-14T17:39:29.236 INFO:journalctl@ceph.mon.a.smithi100.stdout: capacity : 536870912 2024-02-14T17:39:29.236 INFO:journalctl@ceph.mon.a.smithi100.stdout: num_shard_bits : 4 2024-02-14T17:39:29.236 INFO:journalctl@ceph.mon.a.smithi100.stdout: strict_capacity_limit : 0 2024-02-14T17:39:29.236 INFO:journalctl@ceph.mon.a.smithi100.stdout: high_pri_pool_ratio: 0.000 2024-02-14T17:39:29.236 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_cache_compressed: (nil) 2024-02-14T17:39:29.236 INFO:journalctl@ceph.mon.a.smithi100.stdout: persistent_cache: (nil) 2024-02-14T17:39:29.236 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_size: 4096 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_size_deviation: 10 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_restart_interval: 16 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: index_block_restart_interval: 1 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: metadata_block_size: 4096 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: partition_filters: 0 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: use_delta_encoding: 1 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: filter_policy: bloomfilter 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: whole_key_filtering: 1 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: verify_compression: 0 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: read_amp_bytes_per_bit: 0 2024-02-14T17:39:29.237 INFO:journalctl@ceph.mon.a.smithi100.stdout: format_version: 5 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout: enable_index_compression: 1 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout: block_align: 0 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout: max_auto_readahead_size: 262144 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout: prepopulate_block_cache: 0 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout: initial_auto_readahead_size: 8192 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compression: NoCompression 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.num_levels: 7 2024-02-14T17:39:29.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T17:39:29.239 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T17:39:29.240 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T17:39:29.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.table_properties_collectors: 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.inplace_update_support: 0 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T17:39:29.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T17:39:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.bloom_locality: 0 2024-02-14T17:39:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.max_successive_merges: 0 2024-02-14T17:39:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T17:39:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T17:39:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T17:39:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T17:39:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.ttl: 2592000 2024-02-14T17:39:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T17:39:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T17:39:29.243 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.enable_blob_files: false 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.min_blob_size: 0 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8eba6cce-de6c-450c-badd-6fc22472d7a8 2024-02-14T17:39:29.244 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932369173896, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932369174748, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 83267, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 262, "table_properties": {"data_size": 81439, "index_size": 238, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10592, "raw_average_key_size": 47, "raw_value_size": 75451, "raw_average_value_size": 341, "num_data_blocks": 10, "num_entries": 221, "num_filter_entries": 221, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707932369, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8eba6cce-de6c-450c-badd-6fc22472d7a8", "db_session_id": "W6NRKTVP69VKEWBGJOWE", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932369174827, "job": 1, "event": "recovery_finished"} 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557c7329a000 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: DB pointer 0x557c73284000 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** DB Stats ** 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T17:39:29.245 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** Compaction Stats [default] ** 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: L0 2/0 83.15 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: Sum 2/0 83.15 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** Compaction Stats [default] ** 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:39:29.246 INFO:journalctl@ceph.mon.a.smithi100.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 149.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: Cumulative compaction: 0.00 GB write, 28.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: Interval compaction: 0.00 GB write, 28.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T17:39:29.247 INFO:journalctl@ceph.mon.a.smithi100.stdout: Block cache BinnedLRUCache@0x557c731a9090#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: starting mon.a rank 0 at public addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] at bind addrs [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: mon.a@-1(???) e1 preinit fsid d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: mon.a@-1(???).mds e1 new map 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: mon.a@-1(???).mds e1 print_map 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout: e1 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout: legacy client fscid: -1 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout: 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout: No filesystems configured 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T17:39:29.248 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:39:29.249 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:39:29.249 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:39:29.249 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-14T17:39:29.249 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 bash[30100]: 79efe7e1a03d9550542f6899f0b3c280f92592f974c853059781ba8a2c9e0883 2024-02-14T17:39:29.249 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 systemd[1]: Started Ceph mon.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:39:29.249 INFO:teuthology.orchestra.run.smithi100.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-14T17:39:29.480 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-14T17:39:29.480 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: monmap e1: 1 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:39:29.480 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: fsmap 2024-02-14T17:39:29.480 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: osdmap e1: 0 total, 0 up, 0 in 2024-02-14T17:39:29.480 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:29 smithi100 ceph-mon[30138]: mgrmap e1: no daemons active 2024-02-14T17:39:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.689+0000 7fdd7e3b3700 1 Processor -- start 2024-02-14T17:39:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.689+0000 7fdd7e3b3700 1 -- start start 2024-02-14T17:39:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.689+0000 7fdd7e3b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78107550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.689+0000 7fdd7e3b3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78107b20 con 0x7fdd78107130 2024-02-14T17:39:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.689+0000 7fdd77fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.689+0000 7fdd77fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78107550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43302/0 (socket says 172.21.15.100:43302) 2024-02-14T17:39:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.689+0000 7fdd77fff700 1 -- 172.21.15.100:0/1381370494 learned_addr learned my addr 172.21.15.100:0/1381370494 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.690+0000 7fdd77fff700 1 -- 172.21.15.100:0/1381370494 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd78105670 con 0x7fdd78107130 2024-02-14T17:39:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.690+0000 7fdd77fff700 1 --2- 172.21.15.100:0/1381370494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78107550 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdd680098a0 tx=0x7fdd68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dea9f7e35724398 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:30.279 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.690+0000 7fdd76ffd700 1 -- 172.21.15.100:0/1381370494 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd68004160 con 0x7fdd78107130 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.690+0000 7fdd76ffd700 1 -- 172.21.15.100:0/1381370494 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fdd680042c0 con 0x7fdd78107130 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.690+0000 7fdd76ffd700 1 -- 172.21.15.100:0/1381370494 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd680118a0 con 0x7fdd78107130 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.691+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/1381370494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 msgr2=0x7fdd78107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.691+0000 7fdd7e3b3700 1 --2- 172.21.15.100:0/1381370494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78107550 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdd680098a0 tx=0x7fdd68009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.691+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/1381370494 shutdown_connections 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.691+0000 7fdd7e3b3700 1 --2- 172.21.15.100:0/1381370494 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78107550 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.691+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/1381370494 >> 172.21.15.100:0/1381370494 conn(0x7fdd78101030 msgr2=0x7fdd78103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.691+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/1381370494 shutdown_connections 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.691+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/1381370494 wait complete. 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.691+0000 7fdd7e3b3700 1 Processor -- start 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.691+0000 7fdd7e3b3700 1 -- start start 2024-02-14T17:39:30.280 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd7e3b3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd7e3b3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78199c80 con 0x7fdd78107130 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd77fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd77fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43318/0 (socket says 172.21.15.100:43318) 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd77fff700 1 -- 172.21.15.100:0/3507976807 learned_addr learned my addr 172.21.15.100:0/3507976807 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd77fff700 1 -- 172.21.15.100:0/3507976807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68009550 con 0x7fdd78107130 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd77fff700 1 --2- 172.21.15.100:0/3507976807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78199740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdd68003f10 tx=0x7fdd68011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd757fa700 1 -- 172.21.15.100:0/3507976807 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd680036a0 con 0x7fdd78107130 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd757fa700 1 -- 172.21.15.100:0/3507976807 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fdd6801a430 con 0x7fdd78107130 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/3507976807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd78199e80 con 0x7fdd78107130 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd757fa700 1 -- 172.21.15.100:0/3507976807 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd68024410 con 0x7fdd78107130 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.692+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/3507976807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7819a320 con 0x7fdd78107130 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.693+0000 7fdd757fa700 1 -- 172.21.15.100:0/3507976807 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdd68024a60 con 0x7fdd78107130 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.693+0000 7fdd757fa700 1 -- 172.21.15.100:0/3507976807 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdd680193c0 con 0x7fdd78107130 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.693+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/3507976807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd780632a0 con 0x7fdd78107130 2024-02-14T17:39:30.281 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.696+0000 7fdd757fa700 1 -- 172.21.15.100:0/3507976807 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdd6801a5a0 con 0x7fdd78107130 2024-02-14T17:39:30.282 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.762+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/3507976807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fdd7819cc00 con 0x7fdd78107130 2024-02-14T17:39:30.282 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.763+0000 7fdd757fa700 1 -- 172.21.15.100:0/3507976807 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdd68031070 con 0x7fdd78107130 2024-02-14T17:39:30.282 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.764+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/3507976807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 msgr2=0x7fdd78199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:30.282 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.764+0000 7fdd7e3b3700 1 --2- 172.21.15.100:0/3507976807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78199740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdd68003f10 tx=0x7fdd68011940 comp rx=0 tx=0).stop 2024-02-14T17:39:30.282 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.764+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/3507976807 shutdown_connections 2024-02-14T17:39:30.282 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.764+0000 7fdd7e3b3700 1 --2- 172.21.15.100:0/3507976807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdd78107130 0x7fdd78199740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:30.282 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.764+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/3507976807 >> 172.21.15.100:0/3507976807 conn(0x7fdd78101030 msgr2=0x7fdd78101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:30.282 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.764+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/3507976807 shutdown_connections 2024-02-14T17:39:30.282 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:29.764+0000 7fdd7e3b3700 1 -- 172.21.15.100:0/3507976807 wait complete. 2024-02-14T17:39:30.282 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-14T17:39:30.314 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:39:30.314 INFO:teuthology.orchestra.run.smithi100.stdout:Creating mgr... 2024-02-14T17:39:30.314 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-14T17:39:30.314 INFO:teuthology.orchestra.run.smithi100.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-14T17:39:30.555 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.y 2024-02-14T17:39:30.555 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.y.service: Unit ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.y.service not loaded. 2024-02-14T17:39:30.711 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d69a9998-cb5f-11ee-95b9-87774f69a715.target.wants/ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@.service. 2024-02-14T17:39:31.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3507976807' entity='client.admin' 2024-02-14T17:39:31.055 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:30 smithi100 podman[30377]: 2024-02-14 17:39:30.948228477 +0000 UTC m=+0.118424233 container create c60fabca0e5e6273da781a37e120198f3f7eba0c92a152afc21e861929b2335f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:39:31.290 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:39:31.290 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-02-14T17:39:31.297 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:39:31.297 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-02-14T17:39:31.297 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-02-14T17:39:31.297 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to enable service . firewalld.service is not available 2024-02-14T17:39:31.305 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:39:31.305 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-02-14T17:39:31.312 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:39:31.312 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-02-14T17:39:31.312 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-02-14T17:39:31.312 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-14T17:39:31.312 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr to start... 2024-02-14T17:39:31.312 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr... 2024-02-14T17:39:31.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:31 smithi100 podman[30377]: 2024-02-14 17:39:31.167401294 +0000 UTC m=+0.337597047 container init c60fabca0e5e6273da781a37e120198f3f7eba0c92a152afc21e861929b2335f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T17:39:31.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:31 smithi100 podman[30377]: 2024-02-14 17:39:31.17271461 +0000 UTC m=+0.342910365 container start c60fabca0e5e6273da781a37e120198f3f7eba0c92a152afc21e861929b2335f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-14T17:39:31.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:31 smithi100 bash[30377]: c60fabca0e5e6273da781a37e120198f3f7eba0c92a152afc21e861929b2335f 2024-02-14T17:39:31.436 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:31 smithi100 systemd[1]: Started Ceph mgr.y for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:39:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/459774551' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:39:32.151 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:31 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:31.728+0000 7f3da6827200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:39:32.151 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:31 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:31.828+0000 7f3da6827200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:39:32.530 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:32.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:39:32.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "d69a9998-cb5f-11ee-95b9-87774f69a715", 2024-02-14T17:39:32.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:39:32.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:39:32.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:39:32.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:39:32.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:32.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:39:32.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:39:32.531 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:39:32.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:32.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:39:32.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:39:32.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:32.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-14T17:39:32.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:39:32.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:32.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:39:32.532 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:39:32.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:32.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:39:32.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:32.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:39:32.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:39:32.533 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:39:32.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:39:32.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:39:32.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:39:32.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:32.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:39:32.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:39:32.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:39:32.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:39:32.541 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:39:32.542 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:39:23.867527+0000", 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.782+0000 7f3327e45700 1 Processor -- start 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.782+0000 7f3327e45700 1 -- start start 2024-02-14T17:39:32.543 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.782+0000 7f3327e45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.782+0000 7f3327e45700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3320079720 con 0x7f33201064a0 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.782+0000 7f3325be1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.783+0000 7f3325be1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43364/0 (socket says 172.21.15.100:43364) 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.783+0000 7f3325be1700 1 -- 172.21.15.100:0/1590072165 learned_addr learned my addr 172.21.15.100:0/1590072165 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.783+0000 7f3325be1700 1 -- 172.21.15.100:0/1590072165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3320108dd0 con 0x7f33201064a0 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.783+0000 7f3325be1700 1 --2- 172.21.15.100:0/1590072165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320108890 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f331c0098a0 tx=0x7f331c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2854d5d31274c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.783+0000 7f3324bdf700 1 -- 172.21.15.100:0/1590072165 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f331c004160 con 0x7f33201064a0 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.783+0000 7f3324bdf700 1 -- 172.21.15.100:0/1590072165 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f331c0042c0 con 0x7f33201064a0 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.783+0000 7f3324bdf700 1 -- 172.21.15.100:0/1590072165 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f331c0118a0 con 0x7f33201064a0 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.784+0000 7f3327e45700 1 -- 172.21.15.100:0/1590072165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 msgr2=0x7f3320108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.784+0000 7f3327e45700 1 --2- 172.21.15.100:0/1590072165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320108890 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f331c0098a0 tx=0x7f331c009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.784+0000 7f3327e45700 1 -- 172.21.15.100:0/1590072165 shutdown_connections 2024-02-14T17:39:32.544 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.784+0000 7f3327e45700 1 --2- 172.21.15.100:0/1590072165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320108890 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.784+0000 7f3327e45700 1 -- 172.21.15.100:0/1590072165 >> 172.21.15.100:0/1590072165 conn(0x7f33200ffef0 msgr2=0x7f3320102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.784+0000 7f3327e45700 1 -- 172.21.15.100:0/1590072165 shutdown_connections 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.784+0000 7f3327e45700 1 -- 172.21.15.100:0/1590072165 wait complete. 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.785+0000 7f3327e45700 1 Processor -- start 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.785+0000 7f3327e45700 1 -- start start 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.785+0000 7f3327e45700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.785+0000 7f3327e45700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3320199b70 con 0x7f33201064a0 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.785+0000 7f3325be1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.785+0000 7f3325be1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43370/0 (socket says 172.21.15.100:43370) 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.785+0000 7f3325be1700 1 -- 172.21.15.100:0/459774551 learned_addr learned my addr 172.21.15.100:0/459774551 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.785+0000 7f3325be1700 1 -- 172.21.15.100:0/459774551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c009550 con 0x7f33201064a0 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.785+0000 7f3325be1700 1 --2- 172.21.15.100:0/459774551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320199630 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f331c003da0 tx=0x7f331c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.785+0000 7f3316ffd700 1 -- 172.21.15.100:0/459774551 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f331c011930 con 0x7f33201064a0 2024-02-14T17:39:32.545 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.786+0000 7f3316ffd700 1 -- 172.21.15.100:0/459774551 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f331c003b00 con 0x7f33201064a0 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.786+0000 7f3327e45700 1 -- 172.21.15.100:0/459774551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3320199d70 con 0x7f33201064a0 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.786+0000 7f3316ffd700 1 -- 172.21.15.100:0/459774551 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f331c01ad50 con 0x7f33201064a0 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.786+0000 7f3327e45700 1 -- 172.21.15.100:0/459774551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332019a210 con 0x7f33201064a0 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.786+0000 7f3316ffd700 1 -- 172.21.15.100:0/459774551 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f331c02d7d0 con 0x7f33201064a0 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.786+0000 7f3316ffd700 1 -- 172.21.15.100:0/459774551 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f331c024730 con 0x7f33201064a0 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.787+0000 7f3327e45700 1 -- 172.21.15.100:0/459774551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3320109aa0 con 0x7f33201064a0 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.791+0000 7f3316ffd700 1 -- 172.21.15.100:0/459774551 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f331c019db0 con 0x7f33201064a0 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.863+0000 7f3327e45700 1 -- 172.21.15.100:0/459774551 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f332019cbf0 con 0x7f33201064a0 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.863+0000 7f3316ffd700 1 -- 172.21.15.100:0/459774551 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f331c035090 con 0x7f33201064a0 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.864+0000 7f3327e45700 1 -- 172.21.15.100:0/459774551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 msgr2=0x7f3320199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.864+0000 7f3327e45700 1 --2- 172.21.15.100:0/459774551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320199630 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f331c003da0 tx=0x7f331c011e90 comp rx=0 tx=0).stop 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.864+0000 7f3327e45700 1 -- 172.21.15.100:0/459774551 shutdown_connections 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.864+0000 7f3327e45700 1 --2- 172.21.15.100:0/459774551 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33201064a0 0x7f3320199630 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:32.546 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.864+0000 7f3327e45700 1 -- 172.21.15.100:0/459774551 >> 172.21.15.100:0/459774551 conn(0x7f33200ffef0 msgr2=0x7f3320100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:32.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.865+0000 7f3327e45700 1 -- 172.21.15.100:0/459774551 shutdown_connections 2024-02-14T17:39:32.547 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:31.865+0000 7f3327e45700 1 -- 172.21.15.100:0/459774551 wait complete. 2024-02-14T17:39:32.547 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (1/15)... 2024-02-14T17:39:32.807 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:32 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:32.480+0000 7f3da6827200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:39:32.808 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:32 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:32.574+0000 7f3da6827200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:39:32.808 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:32 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:32.712+0000 7f3da6827200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:39:33.150 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:32 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:32.805+0000 7f3da6827200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:39:33.150 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:32 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:32.886+0000 7f3da6827200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:39:33.435 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:33 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:33.148+0000 7f3da6827200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:39:33.435 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:33 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:33.256+0000 7f3da6827200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:39:33.435 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:33 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:33.340+0000 7f3da6827200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:39:33.435 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:33 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:33.433+0000 7f3da6827200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:39:33.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:33 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:33.606+0000 7f3da6827200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:39:34.817 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:34 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:34.562+0000 7f3da6827200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:39:34.818 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:34 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:34.694+0000 7f3da6827200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:39:35.067 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:35 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:35.036+0000 7f3da6827200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:39:35.338 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2305133792' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:39:35.338 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:35 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:35.128+0000 7f3da6827200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:39:35.338 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:35 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:35.336+0000 7f3da6827200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:39:35.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:35 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:35.520+0000 7f3da6827200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:39:35.753 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:35.753 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:39:35.753 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "d69a9998-cb5f-11ee-95b9-87774f69a715", 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:39:35.754 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:39:35.755 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:39:35.764 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:39:35.764 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:39:35.764 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:39:35.764 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:35.764 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:39:35.764 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:39:35.764 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:39:35.764 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:39:35.764 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:39:35.765 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:39:23.867527+0000", 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.061+0000 7fc859d5a700 1 Processor -- start 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.061+0000 7fc859d5a700 1 -- start start 2024-02-14T17:39:35.766 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.061+0000 7fc859d5a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc8541044a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.061+0000 7fc859d5a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854104a70 con 0x7fc854104080 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.061+0000 7fc8537fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc8541044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.061+0000 7fc8537fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc8541044a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57098/0 (socket says 172.21.15.100:57098) 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.061+0000 7fc8537fe700 1 -- 172.21.15.100:0/104457795 learned_addr learned my addr 172.21.15.100:0/104457795 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.062+0000 7fc8537fe700 1 -- 172.21.15.100:0/104457795 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc854105280 con 0x7fc854104080 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.062+0000 7fc8537fe700 1 --2- 172.21.15.100:0/104457795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc8541044a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8480098a0 tx=0x7fc848009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c649a96104cba0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.062+0000 7fc8527fc700 1 -- 172.21.15.100:0/104457795 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc848004160 con 0x7fc854104080 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.062+0000 7fc8527fc700 1 -- 172.21.15.100:0/104457795 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fc8480042c0 con 0x7fc854104080 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.062+0000 7fc8527fc700 1 -- 172.21.15.100:0/104457795 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8480118a0 con 0x7fc854104080 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.063+0000 7fc859d5a700 1 -- 172.21.15.100:0/104457795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 msgr2=0x7fc8541044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.063+0000 7fc859d5a700 1 --2- 172.21.15.100:0/104457795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc8541044a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc8480098a0 tx=0x7fc848009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.063+0000 7fc859d5a700 1 -- 172.21.15.100:0/104457795 shutdown_connections 2024-02-14T17:39:35.767 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.063+0000 7fc859d5a700 1 --2- 172.21.15.100:0/104457795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc8541044a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.063+0000 7fc859d5a700 1 -- 172.21.15.100:0/104457795 >> 172.21.15.100:0/104457795 conn(0x7fc8540ff640 msgr2=0x7fc854101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.063+0000 7fc859d5a700 1 -- 172.21.15.100:0/104457795 shutdown_connections 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.063+0000 7fc859d5a700 1 -- 172.21.15.100:0/104457795 wait complete. 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.063+0000 7fc859d5a700 1 Processor -- start 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.064+0000 7fc859d5a700 1 -- start start 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.064+0000 7fc859d5a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc854199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.064+0000 7fc859d5a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc854199c40 con 0x7fc854104080 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.064+0000 7fc8537fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc854199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.064+0000 7fc8537fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc854199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57106/0 (socket says 172.21.15.100:57106) 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.064+0000 7fc8537fe700 1 -- 172.21.15.100:0/2305133792 learned_addr learned my addr 172.21.15.100:0/2305133792 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.064+0000 7fc8537fe700 1 -- 172.21.15.100:0/2305133792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc848009550 con 0x7fc854104080 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.064+0000 7fc8537fe700 1 --2- 172.21.15.100:0/2305133792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc854199700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc8480046c0 tx=0x7fc848011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.064+0000 7fc850ff9700 1 -- 172.21.15.100:0/2305133792 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8480036a0 con 0x7fc854104080 2024-02-14T17:39:35.768 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.064+0000 7fc859d5a700 1 -- 172.21.15.100:0/2305133792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc854199e40 con 0x7fc854104080 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.065+0000 7fc850ff9700 1 -- 172.21.15.100:0/2305133792 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fc84801a430 con 0x7fc854104080 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.065+0000 7fc850ff9700 1 -- 172.21.15.100:0/2305133792 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8480193c0 con 0x7fc854104080 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.065+0000 7fc859d5a700 1 -- 172.21.15.100:0/2305133792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85419a2e0 con 0x7fc854104080 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.065+0000 7fc850ff9700 1 -- 172.21.15.100:0/2305133792 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc8480199d0 con 0x7fc854104080 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.065+0000 7fc850ff9700 1 -- 172.21.15.100:0/2305133792 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc84802de40 con 0x7fc854104080 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.066+0000 7fc859d5a700 1 -- 172.21.15.100:0/2305133792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc854191160 con 0x7fc854104080 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.069+0000 7fc850ff9700 1 -- 172.21.15.100:0/2305133792 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc84801a5a0 con 0x7fc854104080 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.141+0000 7fc859d5a700 1 -- 172.21.15.100:0/2305133792 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc8541058d0 con 0x7fc854104080 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.141+0000 7fc850ff9700 1 -- 172.21.15.100:0/2305133792 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fc84801a8f0 con 0x7fc854104080 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.142+0000 7fc859d5a700 1 -- 172.21.15.100:0/2305133792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 msgr2=0x7fc854199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.142+0000 7fc859d5a700 1 --2- 172.21.15.100:0/2305133792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc854199700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc8480046c0 tx=0x7fc848011eb0 comp rx=0 tx=0).stop 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.142+0000 7fc859d5a700 1 -- 172.21.15.100:0/2305133792 shutdown_connections 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.142+0000 7fc859d5a700 1 --2- 172.21.15.100:0/2305133792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc854104080 0x7fc854199700 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.142+0000 7fc859d5a700 1 -- 172.21.15.100:0/2305133792 >> 172.21.15.100:0/2305133792 conn(0x7fc8540ff640 msgr2=0x7fc85418e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:35.769 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.142+0000 7fc859d5a700 1 -- 172.21.15.100:0/2305133792 shutdown_connections 2024-02-14T17:39:35.770 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:35.142+0000 7fc859d5a700 1 -- 172.21.15.100:0/2305133792 wait complete. 2024-02-14T17:39:35.770 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (2/15)... 2024-02-14T17:39:36.058 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:35 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:35.802+0000 7f3da6827200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:39:36.058 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:36 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:36.056+0000 7f3da6827200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:39:36.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:36 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:36.292+0000 7f3da6827200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:39:37.150 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:37 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:37.091+0000 7f3da6827200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:39:37.980 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:37 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:37.653+0000 7f3da6827200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:39:37.980 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:37 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:37.749+0000 7f3da6827200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:39:37.980 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:37 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:37.978+0000 7f3da6827200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: Activating manager daemon y 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: mgrmap e2: y(active, starting, since 0.00339669s) 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: from='mgr.14100 172.21.15.100:0/718843310' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: from='mgr.14100 172.21.15.100:0/718843310' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: from='mgr.14100 172.21.15.100:0/718843310' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: from='mgr.14100 172.21.15.100:0/718843310' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: from='mgr.14100 172.21.15.100:0/718843310' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: Manager daemon y is now available 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: from='mgr.14100 172.21.15.100:0/718843310' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: from='mgr.14100 172.21.15.100:0/718843310' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:39:38.303 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: from='mgr.14100 172.21.15.100:0/718843310' entity='mgr.y' 2024-02-14T17:39:38.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: from='mgr.14100 172.21.15.100:0/718843310' entity='mgr.y' 2024-02-14T17:39:38.304 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:38 smithi100 ceph-mon[30138]: from='mgr.14100 172.21.15.100:0/718843310' entity='mgr.y' 2024-02-14T17:39:38.883 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "d69a9998-cb5f-11ee-95b9-87774f69a715", 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:39:38.884 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:39:38.885 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:39:38.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:39:38.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:39:38.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:39:38.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:38.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:39:38.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:39:38.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:39:38.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:39:38.893 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:39:38.894 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:39:23.867527+0000", 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.217+0000 7f71fa415700 1 Processor -- start 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.217+0000 7f71fa415700 1 -- start start 2024-02-14T17:39:38.895 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.217+0000 7f71fa415700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.217+0000 7f71fa415700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40796e0 con 0x7f71f41064a0 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.217+0000 7f71f3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.218+0000 7f71f3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57168/0 (socket says 172.21.15.100:57168) 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.218+0000 7f71f3fff700 1 -- 172.21.15.100:0/1951194345 learned_addr learned my addr 172.21.15.100:0/1951194345 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.218+0000 7f71f3fff700 1 -- 172.21.15.100:0/1951194345 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f4108dd0 con 0x7f71f41064a0 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.218+0000 7f71f3fff700 1 --2- 172.21.15.100:0/1951194345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4108890 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f71e40098a0 tx=0x7f71e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd8355299cc666d server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.218+0000 7f71f2ffd700 1 -- 172.21.15.100:0/1951194345 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71e4004160 con 0x7f71f41064a0 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.218+0000 7f71f2ffd700 1 -- 172.21.15.100:0/1951194345 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f71e40042c0 con 0x7f71f41064a0 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.218+0000 7f71f2ffd700 1 -- 172.21.15.100:0/1951194345 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71e40118a0 con 0x7f71f41064a0 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.219+0000 7f71fa415700 1 -- 172.21.15.100:0/1951194345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 msgr2=0x7f71f4108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.219+0000 7f71fa415700 1 --2- 172.21.15.100:0/1951194345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4108890 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f71e40098a0 tx=0x7f71e4009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.219+0000 7f71fa415700 1 -- 172.21.15.100:0/1951194345 shutdown_connections 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.219+0000 7f71fa415700 1 --2- 172.21.15.100:0/1951194345 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4108890 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.219+0000 7f71fa415700 1 -- 172.21.15.100:0/1951194345 >> 172.21.15.100:0/1951194345 conn(0x7f71f40ffe50 msgr2=0x7f71f41022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:38.896 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.219+0000 7f71fa415700 1 -- 172.21.15.100:0/1951194345 shutdown_connections 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.219+0000 7f71fa415700 1 -- 172.21.15.100:0/1951194345 wait complete. 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.220+0000 7f71fa415700 1 Processor -- start 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.220+0000 7f71fa415700 1 -- start start 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.220+0000 7f71fa415700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.220+0000 7f71fa415700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f4199b90 con 0x7f71f41064a0 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.220+0000 7f71f3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.220+0000 7f71f3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57178/0 (socket says 172.21.15.100:57178) 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.220+0000 7f71f3fff700 1 -- 172.21.15.100:0/1821900347 learned_addr learned my addr 172.21.15.100:0/1821900347 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.220+0000 7f71f3fff700 1 -- 172.21.15.100:0/1821900347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e4009550 con 0x7f71f41064a0 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.220+0000 7f71f3fff700 1 --2- 172.21.15.100:0/1821900347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4199650 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f71e4011890 tx=0x7f71e4011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.221+0000 7f71f17fa700 1 -- 172.21.15.100:0/1821900347 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71e40036a0 con 0x7f71f41064a0 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.221+0000 7f71fa415700 1 -- 172.21.15.100:0/1821900347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f4199d90 con 0x7f71f41064a0 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.221+0000 7f71f17fa700 1 -- 172.21.15.100:0/1821900347 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f71e401a430 con 0x7f71f41064a0 2024-02-14T17:39:38.897 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.221+0000 7f71f17fa700 1 -- 172.21.15.100:0/1821900347 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71e40193c0 con 0x7f71f41064a0 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.221+0000 7f71fa415700 1 -- 172.21.15.100:0/1821900347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f419a230 con 0x7f71f41064a0 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.221+0000 7f71f17fa700 1 -- 172.21.15.100:0/1821900347 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 2) v1 ==== 45688+0+0 (secure 0 0 0) 0x7f71e4019640 con 0x7f71f41064a0 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.222+0000 7f71f17fa700 1 -- 172.21.15.100:0/1821900347 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f71e404d9e0 con 0x7f71f41064a0 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.222+0000 7f71fa415700 1 -- 172.21.15.100:0/1821900347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f4109a40 con 0x7f71f41064a0 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.224+0000 7f71f17fa700 1 -- 172.21.15.100:0/1821900347 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f71e4018940 con 0x7f71f41064a0 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.300+0000 7f71fa415700 1 -- 172.21.15.100:0/1821900347 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f71f419cb70 con 0x7f71f41064a0 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.301+0000 7f71f17fa700 1 -- 172.21.15.100:0/1821900347 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f71e4018350 con 0x7f71f41064a0 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.302+0000 7f71fa415700 1 -- 172.21.15.100:0/1821900347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 msgr2=0x7f71f4199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.302+0000 7f71fa415700 1 --2- 172.21.15.100:0/1821900347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4199650 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f71e4011890 tx=0x7f71e4011970 comp rx=0 tx=0).stop 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.302+0000 7f71fa415700 1 -- 172.21.15.100:0/1821900347 shutdown_connections 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.302+0000 7f71fa415700 1 --2- 172.21.15.100:0/1821900347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71f41064a0 0x7f71f4199650 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.302+0000 7f71fa415700 1 -- 172.21.15.100:0/1821900347 >> 172.21.15.100:0/1821900347 conn(0x7f71f40ffe50 msgr2=0x7f71f4100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.302+0000 7f71fa415700 1 -- 172.21.15.100:0/1821900347 shutdown_connections 2024-02-14T17:39:38.898 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:38.302+0000 7f71fa415700 1 -- 172.21.15.100:0/1821900347 wait complete. 2024-02-14T17:39:38.899 INFO:teuthology.orchestra.run.smithi100.stdout:mgr not available, waiting (3/15)... 2024-02-14T17:39:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1821900347' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:39:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:39 smithi100 ceph-mon[30138]: mgrmap e3: y(active, since 1.0064s) 2024-02-14T17:39:41.347 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:41 smithi100 ceph-mon[30138]: mgrmap e4: y(active, since 2s) 2024-02-14T17:39:42.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2507938449' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsid": "d69a9998-cb5f-11ee-95b9-87774f69a715", 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "health": { 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 0 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:42.206 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "a" 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-14T17:39:42.207 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-14T17:39:42.215 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-14T17:39:42.215 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-14T17:39:42.216 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "restful" 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ], 2024-02-14T17:39:42.217 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "modified": "2024-02-14T17:39:23.867527+0000", 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout }, 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.358+0000 7f6caf3d4700 1 Processor -- start 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.358+0000 7f6caf3d4700 1 -- start start 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.358+0000 7f6caf3d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8104110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.358+0000 7f6caf3d4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca81046e0 con 0x7f6ca8103cf0 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.358+0000 7f6cad170700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8104110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.359+0000 7f6cad170700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8104110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57192/0 (socket says 172.21.15.100:57192) 2024-02-14T17:39:42.218 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.359+0000 7f6cad170700 1 -- 172.21.15.100:0/3201478671 learned_addr learned my addr 172.21.15.100:0/3201478671 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.359+0000 7f6cad170700 1 -- 172.21.15.100:0/3201478671 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8106c80 con 0x7f6ca8103cf0 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.359+0000 7f6cad170700 1 --2- 172.21.15.100:0/3201478671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8104110 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009e80 tx=0x7f6ca400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b61b12ca7e904d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.359+0000 7f6c9ffff700 1 -- 172.21.15.100:0/3201478671 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ca400e070 con 0x7f6ca8103cf0 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.359+0000 7f6c9ffff700 1 -- 172.21.15.100:0/3201478671 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f6ca4003e80 con 0x7f6ca8103cf0 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.359+0000 7f6c9ffff700 1 -- 172.21.15.100:0/3201478671 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ca4004370 con 0x7f6ca8103cf0 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 -- 172.21.15.100:0/3201478671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 msgr2=0x7f6ca8104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 --2- 172.21.15.100:0/3201478671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8104110 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4009e80 tx=0x7f6ca400b330 comp rx=0 tx=0).stop 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 -- 172.21.15.100:0/3201478671 shutdown_connections 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 --2- 172.21.15.100:0/3201478671 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8104110 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 -- 172.21.15.100:0/3201478671 >> 172.21.15.100:0/3201478671 conn(0x7f6ca80ff640 msgr2=0x7f6ca8101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 -- 172.21.15.100:0/3201478671 shutdown_connections 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 -- 172.21.15.100:0/3201478671 wait complete. 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 Processor -- start 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 -- start start 2024-02-14T17:39:42.219 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8106500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.360+0000 7f6caf3d4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8106a40 con 0x7f6ca8103cf0 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.361+0000 7f6cad170700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8106500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.361+0000 7f6cad170700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8106500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57194/0 (socket says 172.21.15.100:57194) 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.361+0000 7f6cad170700 1 -- 172.21.15.100:0/2507938449 learned_addr learned my addr 172.21.15.100:0/2507938449 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.361+0000 7f6cad170700 1 -- 172.21.15.100:0/2507938449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4009b30 con 0x7f6ca8103cf0 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.361+0000 7f6cad170700 1 --2- 172.21.15.100:0/2507938449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8106500 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4004000 tx=0x7f6ca40048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.361+0000 7f6c9e7fc700 1 -- 172.21.15.100:0/2507938449 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ca400e040 con 0x7f6ca8103cf0 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.361+0000 7f6c9e7fc700 1 -- 172.21.15.100:0/2507938449 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f6ca400b8c0 con 0x7f6ca8103cf0 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.361+0000 7f6caf3d4700 1 -- 172.21.15.100:0/2507938449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca8104c30 con 0x7f6ca8103cf0 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.361+0000 7f6c9e7fc700 1 -- 172.21.15.100:0/2507938449 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ca4012720 con 0x7f6ca8103cf0 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.361+0000 7f6caf3d4700 1 -- 172.21.15.100:0/2507938449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca8105130 con 0x7f6ca8103cf0 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.362+0000 7f6c9e7fc700 1 -- 172.21.15.100:0/2507938449 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f6ca4012990 con 0x7f6ca8103cf0 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.362+0000 7f6c9e7fc700 1 --2- 172.21.15.100:0/2507938449 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f6c94038ee0 0x7f6c9403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:42.220 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.362+0000 7f6caf3d4700 1 -- 172.21.15.100:0/2507938449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca80632a0 con 0x7f6ca8103cf0 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.362+0000 7f6c9e7fc700 1 -- 172.21.15.100:0/2507938449 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6ca404cef0 con 0x7f6ca8103cf0 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.362+0000 7f6cac96f700 1 --2- 172.21.15.100:0/2507938449 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f6c94038ee0 0x7f6c9403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.363+0000 7f6cac96f700 1 --2- 172.21.15.100:0/2507938449 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f6c94038ee0 0x7f6c9403b3a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6c98006fd0 tx=0x7f6c98006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.367+0000 7f6c9e7fc700 1 -- 172.21.15.100:0/2507938449 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ca4025400 con 0x7f6ca8103cf0 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.636+0000 7f6caf3d4700 1 -- 172.21.15.100:0/2507938449 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6ca8109eb0 con 0x7f6ca8103cf0 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.636+0000 7f6c9e7fc700 1 -- 172.21.15.100:0/2507938449 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f6ca4020020 con 0x7f6ca8103cf0 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.639+0000 7f6caf3d4700 1 -- 172.21.15.100:0/2507938449 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f6c94038ee0 msgr2=0x7f6c9403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.639+0000 7f6caf3d4700 1 --2- 172.21.15.100:0/2507938449 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f6c94038ee0 0x7f6c9403b3a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6c98006fd0 tx=0x7f6c98006e40 comp rx=0 tx=0).stop 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.639+0000 7f6caf3d4700 1 -- 172.21.15.100:0/2507938449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 msgr2=0x7f6ca8106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.639+0000 7f6caf3d4700 1 --2- 172.21.15.100:0/2507938449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8106500 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4004000 tx=0x7f6ca40048d0 comp rx=0 tx=0).stop 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.639+0000 7f6caf3d4700 1 -- 172.21.15.100:0/2507938449 shutdown_connections 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.639+0000 7f6caf3d4700 1 --2- 172.21.15.100:0/2507938449 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f6c94038ee0 0x7f6c9403b3a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.639+0000 7f6caf3d4700 1 --2- 172.21.15.100:0/2507938449 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ca8103cf0 0x7f6ca8106500 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.639+0000 7f6caf3d4700 1 -- 172.21.15.100:0/2507938449 >> 172.21.15.100:0/2507938449 conn(0x7f6ca80ff640 msgr2=0x7f6ca8100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.639+0000 7f6caf3d4700 1 -- 172.21.15.100:0/2507938449 shutdown_connections 2024-02-14T17:39:42.221 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:41.640+0000 7f6caf3d4700 1 -- 172.21.15.100:0/2507938449 wait complete. 2024-02-14T17:39:42.222 INFO:teuthology.orchestra.run.smithi100.stdout:mgr is available 2024-02-14T17:39:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/941568570' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-14T17:39:43.474 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:43.474 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [global] 2024-02-14T17:39:43.474 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout fsid = d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:43.474 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-14T17:39:43.474 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-14T17:39:43.474 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-14T17:39:43.474 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-14T17:39:43.474 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout [osd] 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.686+0000 7f0f62620700 1 Processor -- start 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.686+0000 7f0f62620700 1 -- start start 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.686+0000 7f0f62620700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c106440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.686+0000 7f0f62620700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c106a10 con 0x7f0f5c106020 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.686+0000 7f0f5bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c106440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.686+0000 7f0f5bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c106440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33206/0 (socket says 172.21.15.100:33206) 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.686+0000 7f0f5bfff700 1 -- 172.21.15.100:0/1349128557 learned_addr learned my addr 172.21.15.100:0/1349128557 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f5bfff700 1 -- 172.21.15.100:0/1349128557 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c104560 con 0x7f0f5c106020 2024-02-14T17:39:43.475 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f5bfff700 1 --2- 172.21.15.100:0/1349128557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c106440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0098a0 tx=0x7f0f4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4b1892674b71d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f5affd700 1 -- 172.21.15.100:0/1349128557 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f4c004160 con 0x7f0f5c106020 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f5affd700 1 -- 172.21.15.100:0/1349128557 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f0f4c0042c0 con 0x7f0f5c106020 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f5affd700 1 -- 172.21.15.100:0/1349128557 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f4c0118a0 con 0x7f0f5c106020 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f62620700 1 -- 172.21.15.100:0/1349128557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 msgr2=0x7f0f5c106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f62620700 1 --2- 172.21.15.100:0/1349128557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c106440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0098a0 tx=0x7f0f4c009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f62620700 1 -- 172.21.15.100:0/1349128557 shutdown_connections 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f62620700 1 --2- 172.21.15.100:0/1349128557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c106440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f62620700 1 -- 172.21.15.100:0/1349128557 >> 172.21.15.100:0/1349128557 conn(0x7f0f5c0fff20 msgr2=0x7f0f5c102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f62620700 1 -- 172.21.15.100:0/1349128557 shutdown_connections 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.687+0000 7f0f62620700 1 -- 172.21.15.100:0/1349128557 wait complete. 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.688+0000 7f0f62620700 1 Processor -- start 2024-02-14T17:39:43.476 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.688+0000 7f0f62620700 1 -- start start 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.688+0000 7f0f62620700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.688+0000 7f0f62620700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c197a90 con 0x7f0f5c106020 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.688+0000 7f0f5bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.688+0000 7f0f5bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33220/0 (socket says 172.21.15.100:33220) 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.688+0000 7f0f5bfff700 1 -- 172.21.15.100:0/941568570 learned_addr learned my addr 172.21.15.100:0/941568570 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.689+0000 7f0f5bfff700 1 -- 172.21.15.100:0/941568570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c009550 con 0x7f0f5c106020 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.689+0000 7f0f5bfff700 1 --2- 172.21.15.100:0/941568570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c197550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c003da0 tx=0x7f0f4c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.689+0000 7f0f597fa700 1 -- 172.21.15.100:0/941568570 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f4c011930 con 0x7f0f5c106020 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.689+0000 7f0f597fa700 1 -- 172.21.15.100:0/941568570 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f0f4c003b00 con 0x7f0f5c106020 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.689+0000 7f0f597fa700 1 -- 172.21.15.100:0/941568570 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f4c01ad50 con 0x7f0f5c106020 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.689+0000 7f0f62620700 1 -- 172.21.15.100:0/941568570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c197c90 con 0x7f0f5c106020 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.689+0000 7f0f62620700 1 -- 172.21.15.100:0/941568570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c198130 con 0x7f0f5c106020 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.690+0000 7f0f597fa700 1 -- 172.21.15.100:0/941568570 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f0f4c02d440 con 0x7f0f5c106020 2024-02-14T17:39:43.477 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.690+0000 7f0f597fa700 1 --2- 172.21.15.100:0/941568570 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f0f4403d2a0 0x7f0f4403f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.690+0000 7f0f62620700 1 -- 172.21.15.100:0/941568570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c0632a0 con 0x7f0f5c106020 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.690+0000 7f0f597fa700 1 -- 172.21.15.100:0/941568570 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0f4c04cd90 con 0x7f0f5c106020 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.690+0000 7f0f5b7fe700 1 --2- 172.21.15.100:0/941568570 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f0f4403d2a0 0x7f0f4403f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.690+0000 7f0f5b7fe700 1 --2- 172.21.15.100:0/941568570 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f0f4403d2a0 0x7f0f4403f760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f50006fd0 tx=0x7f0f50006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.695+0000 7f0f597fa700 1 -- 172.21.15.100:0/941568570 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f4c02d6f0 con 0x7f0f5c106020 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.884+0000 7f0f62620700 1 -- 172.21.15.100:0/941568570 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0f5c078840 con 0x7f0f5c106020 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.885+0000 7f0f597fa700 1 -- 172.21.15.100:0/941568570 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+282 (secure 0 0 0) 0x7f0f4c018350 con 0x7f0f5c106020 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.887+0000 7f0f62620700 1 -- 172.21.15.100:0/941568570 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f0f4403d2a0 msgr2=0x7f0f4403f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.887+0000 7f0f62620700 1 --2- 172.21.15.100:0/941568570 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f0f4403d2a0 0x7f0f4403f760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0f50006fd0 tx=0x7f0f50006e40 comp rx=0 tx=0).stop 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.887+0000 7f0f62620700 1 -- 172.21.15.100:0/941568570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 msgr2=0x7f0f5c197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.887+0000 7f0f62620700 1 --2- 172.21.15.100:0/941568570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c197550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c003da0 tx=0x7f0f4c011e90 comp rx=0 tx=0).stop 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.887+0000 7f0f62620700 1 -- 172.21.15.100:0/941568570 shutdown_connections 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.887+0000 7f0f62620700 1 --2- 172.21.15.100:0/941568570 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f0f4403d2a0 0x7f0f4403f760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.887+0000 7f0f62620700 1 --2- 172.21.15.100:0/941568570 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f5c106020 0x7f0f5c197550 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:43.478 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.887+0000 7f0f62620700 1 -- 172.21.15.100:0/941568570 >> 172.21.15.100:0/941568570 conn(0x7f0f5c0fff20 msgr2=0x7f0f5c18dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:43.479 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.888+0000 7f0f62620700 1 -- 172.21.15.100:0/941568570 shutdown_connections 2024-02-14T17:39:43.479 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:42.888+0000 7f0f62620700 1 -- 172.21.15.100:0/941568570 wait complete. 2024-02-14T17:39:43.479 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling cephadm module... 2024-02-14T17:39:44.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3674995697' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-14T17:39:44.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ignoring --setuser ceph since I am not root 2024-02-14T17:39:44.401 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ignoring --setgroup ceph since I am not root 2024-02-14T17:39:44.955 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:44.766+0000 7f21ff63f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:39:44.955 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:44.859+0000 7f21ff63f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:39:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.891+0000 7f1680274700 1 Processor -- start 2024-02-14T17:39:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.891+0000 7f1680274700 1 -- start start 2024-02-14T17:39:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.892+0000 7f1680274700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f16781088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.892+0000 7f1680274700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780796c0 con 0x7f1678106500 2024-02-14T17:39:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.892+0000 7f167e010700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f16781088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.892+0000 7f167e010700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f16781088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33224/0 (socket says 172.21.15.100:33224) 2024-02-14T17:39:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.892+0000 7f167e010700 1 -- 172.21.15.100:0/2883719827 learned_addr learned my addr 172.21.15.100:0/2883719827 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.892+0000 7f167e010700 1 -- 172.21.15.100:0/2883719827 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678108e30 con 0x7f1678106500 2024-02-14T17:39:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.892+0000 7f167e010700 1 --2- 172.21.15.100:0/2883719827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f16781088f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f16740098a0 tx=0x7f1674009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ba5d9531c43c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:44.968 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.892+0000 7f167d00e700 1 -- 172.21.15.100:0/2883719827 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1674004160 con 0x7f1678106500 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.892+0000 7f167d00e700 1 -- 172.21.15.100:0/2883719827 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f16740042c0 con 0x7f1678106500 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.892+0000 7f167d00e700 1 -- 172.21.15.100:0/2883719827 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16740118a0 con 0x7f1678106500 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.893+0000 7f1680274700 1 -- 172.21.15.100:0/2883719827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 msgr2=0x7f16781088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.893+0000 7f1680274700 1 --2- 172.21.15.100:0/2883719827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f16781088f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f16740098a0 tx=0x7f1674009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.893+0000 7f1680274700 1 -- 172.21.15.100:0/2883719827 shutdown_connections 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.893+0000 7f1680274700 1 --2- 172.21.15.100:0/2883719827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f16781088f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.893+0000 7f1680274700 1 -- 172.21.15.100:0/2883719827 >> 172.21.15.100:0/2883719827 conn(0x7f16780ffef0 msgr2=0x7f1678102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.893+0000 7f1680274700 1 -- 172.21.15.100:0/2883719827 shutdown_connections 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.893+0000 7f1680274700 1 -- 172.21.15.100:0/2883719827 wait complete. 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.893+0000 7f1680274700 1 Processor -- start 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f1680274700 1 -- start start 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f1680274700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f1678105d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f1680274700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16781062c0 con 0x7f1678106500 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f167e010700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f1678105d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:44.969 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f167e010700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f1678105d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33230/0 (socket says 172.21.15.100:33230) 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f167e010700 1 -- 172.21.15.100:0/3674995697 learned_addr learned my addr 172.21.15.100:0/3674995697 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f167e010700 1 -- 172.21.15.100:0/3674995697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1674009550 con 0x7f1678106500 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f167e010700 1 --2- 172.21.15.100:0/3674995697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f1678105d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1674011890 tx=0x7f1674011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f166f7fe700 1 -- 172.21.15.100:0/3674995697 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16740036a0 con 0x7f1678106500 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f166f7fe700 1 -- 172.21.15.100:0/3674995697 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f167401a430 con 0x7f1678106500 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f1680274700 1 -- 172.21.15.100:0/3674995697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16781044b0 con 0x7f1678106500 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f166f7fe700 1 -- 172.21.15.100:0/3674995697 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16740193c0 con 0x7f1678106500 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.894+0000 7f1680274700 1 -- 172.21.15.100:0/3674995697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1678104950 con 0x7f1678106500 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.895+0000 7f166f7fe700 1 -- 172.21.15.100:0/3674995697 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f16740195f0 con 0x7f1678106500 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.895+0000 7f166f7fe700 1 --2- 172.21.15.100:0/3674995697 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1664038ee0 0x7f166403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.895+0000 7f166f7fe700 1 -- 172.21.15.100:0/3674995697 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f167404cee0 con 0x7f1678106500 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.895+0000 7f1680274700 1 -- 172.21.15.100:0/3674995697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16780632a0 con 0x7f1678106500 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.895+0000 7f167d80f700 1 --2- 172.21.15.100:0/3674995697 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1664038ee0 0x7f166403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:44.970 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.896+0000 7f167d80f700 1 --2- 172.21.15.100:0/3674995697 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1664038ee0 0x7f166403b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1668006fd0 tx=0x7f1668006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:43.900+0000 7f166f7fe700 1 -- 172.21.15.100:0/3674995697 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1674018940 con 0x7f1678106500 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.147+0000 7f1680274700 1 -- 172.21.15.100:0/3674995697 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f1678109c70 con 0x7f1678106500 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.198+0000 7f166f7fe700 1 -- 172.21.15.100:0/3674995697 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f1674018350 con 0x7f1678106500 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.200+0000 7f166f7fe700 1 -- 172.21.15.100:0/3674995697 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f167401a5a0 con 0x7f1678106500 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.201+0000 7f1680274700 1 -- 172.21.15.100:0/3674995697 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1664038ee0 msgr2=0x7f166403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.201+0000 7f1680274700 1 --2- 172.21.15.100:0/3674995697 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1664038ee0 0x7f166403b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1668006fd0 tx=0x7f1668006e40 comp rx=0 tx=0).stop 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.201+0000 7f1680274700 1 -- 172.21.15.100:0/3674995697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 msgr2=0x7f1678105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.201+0000 7f1680274700 1 --2- 172.21.15.100:0/3674995697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f1678105d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1674011890 tx=0x7f1674011970 comp rx=0 tx=0).stop 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.201+0000 7f1680274700 1 -- 172.21.15.100:0/3674995697 shutdown_connections 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.201+0000 7f1680274700 1 --2- 172.21.15.100:0/3674995697 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1664038ee0 0x7f166403b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.201+0000 7f1680274700 1 --2- 172.21.15.100:0/3674995697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1678106500 0x7f1678105d80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.201+0000 7f1680274700 1 -- 172.21.15.100:0/3674995697 >> 172.21.15.100:0/3674995697 conn(0x7f16780ffef0 msgr2=0x7f16781009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.201+0000 7f1680274700 1 -- 172.21.15.100:0/3674995697 shutdown_connections 2024-02-14T17:39:44.971 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:44.201+0000 7f1680274700 1 -- 172.21.15.100:0/3674995697 wait complete. 2024-02-14T17:39:45.220 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3674995697' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-14T17:39:45.220 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:45 smithi100 ceph-mon[30138]: mgrmap e5: y(active, since 6s) 2024-02-14T17:39:45.854 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:45 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:45.516+0000 7f21ff63f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:39:45.854 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:45 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:45.610+0000 7f21ff63f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:39:45.855 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:45 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:45.757+0000 7f21ff63f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:39:45.855 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:45 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:45.852+0000 7f21ff63f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:39:46.150 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:45 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:45.935+0000 7f21ff63f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:39:46.361 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.458+0000 7f3560b85700 1 Processor -- start 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.458+0000 7f3560b85700 1 -- start start 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.458+0000 7f3560b85700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.458+0000 7f3560b85700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c079720 con 0x7f355c1064a0 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.458+0000 7f355a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.458+0000 7f355a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33252/0 (socket says 172.21.15.100:33252) 2024-02-14T17:39:46.362 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.458+0000 7f355a59c700 1 -- 172.21.15.100:0/1201151274 learned_addr learned my addr 172.21.15.100:0/1201151274 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.458+0000 7f355a59c700 1 -- 172.21.15.100:0/1201151274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c108dd0 con 0x7f355c1064a0 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.459+0000 7f355a59c700 1 --2- 172.21.15.100:0/1201151274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c108890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35440098a0 tx=0x7f3544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24da5cb5e4905610 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.459+0000 7f355959a700 1 -- 172.21.15.100:0/1201151274 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3544004160 con 0x7f355c1064a0 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.459+0000 7f355959a700 1 -- 172.21.15.100:0/1201151274 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f35440042c0 con 0x7f355c1064a0 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.459+0000 7f355959a700 1 -- 172.21.15.100:0/1201151274 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35440118a0 con 0x7f355c1064a0 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.459+0000 7f3560b85700 1 -- 172.21.15.100:0/1201151274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 msgr2=0x7f355c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.459+0000 7f3560b85700 1 --2- 172.21.15.100:0/1201151274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c108890 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35440098a0 tx=0x7f3544009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.459+0000 7f3560b85700 1 -- 172.21.15.100:0/1201151274 shutdown_connections 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.459+0000 7f3560b85700 1 --2- 172.21.15.100:0/1201151274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c108890 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.459+0000 7f3560b85700 1 -- 172.21.15.100:0/1201151274 >> 172.21.15.100:0/1201151274 conn(0x7f355c0ffef0 msgr2=0x7f355c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.460+0000 7f3560b85700 1 -- 172.21.15.100:0/1201151274 shutdown_connections 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.460+0000 7f3560b85700 1 -- 172.21.15.100:0/1201151274 wait complete. 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.460+0000 7f3560b85700 1 Processor -- start 2024-02-14T17:39:46.363 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.460+0000 7f3560b85700 1 -- start start 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.460+0000 7f3560b85700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c105d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.460+0000 7f3560b85700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c106260 con 0x7f355c1064a0 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.460+0000 7f355a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.461+0000 7f355a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33266/0 (socket says 172.21.15.100:33266) 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.461+0000 7f355a59c700 1 -- 172.21.15.100:0/1851372948 learned_addr learned my addr 172.21.15.100:0/1851372948 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.461+0000 7f355a59c700 1 -- 172.21.15.100:0/1851372948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544009550 con 0x7f355c1064a0 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.461+0000 7f355a59c700 1 --2- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c105d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3544011890 tx=0x7f3544011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.461+0000 7f35537fe700 1 -- 172.21.15.100:0/1851372948 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35440036a0 con 0x7f355c1064a0 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.461+0000 7f35537fe700 1 -- 172.21.15.100:0/1851372948 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f354401a430 con 0x7f355c1064a0 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.461+0000 7f3560b85700 1 -- 172.21.15.100:0/1851372948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c104450 con 0x7f355c1064a0 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.461+0000 7f35537fe700 1 -- 172.21.15.100:0/1851372948 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35440193c0 con 0x7f355c1064a0 2024-02-14T17:39:46.364 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.461+0000 7f3560b85700 1 -- 172.21.15.100:0/1851372948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c1048f0 con 0x7f355c1064a0 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.462+0000 7f35537fe700 1 -- 172.21.15.100:0/1851372948 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f3544019640 con 0x7f355c1064a0 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.462+0000 7f35537fe700 1 --2- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f3548038ee0 0x7f354803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.462+0000 7f35537fe700 1 -- 172.21.15.100:0/1851372948 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f354404dec0 con 0x7f355c1064a0 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.462+0000 7f3559d9b700 1 -- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f3548038ee0 msgr2=0x7f354803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2036660321 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.462+0000 7f3559d9b700 1 --2- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f3548038ee0 0x7f354803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.462+0000 7f3560b85700 1 -- 172.21.15.100:0/1851372948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c0632a0 con 0x7f355c1064a0 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.467+0000 7f35537fe700 1 -- 172.21.15.100:0/1851372948 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3544018940 con 0x7f355c1064a0 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.663+0000 7f3559d9b700 1 -- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f3548038ee0 msgr2=0x7f354803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2036660321 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.663+0000 7f3559d9b700 1 --2- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f3548038ee0 0x7f354803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.722+0000 7f3560b85700 1 -- 172.21.15.100:0/1851372948 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f355c109c50 con 0x7f355c1064a0 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.723+0000 7f35537fe700 1 -- 172.21.15.100:0/1851372948 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3544018350 con 0x7f355c1064a0 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.725+0000 7f3560b85700 1 -- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f3548038ee0 msgr2=0x7f354803b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.725+0000 7f3560b85700 1 --2- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f3548038ee0 0x7f354803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:46.365 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.725+0000 7f3560b85700 1 -- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 msgr2=0x7f355c105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:46.366 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.725+0000 7f3560b85700 1 --2- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c105d20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3544011890 tx=0x7f3544011970 comp rx=0 tx=0).stop 2024-02-14T17:39:46.366 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.725+0000 7f3560b85700 1 -- 172.21.15.100:0/1851372948 shutdown_connections 2024-02-14T17:39:46.366 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.725+0000 7f3560b85700 1 --2- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f3548038ee0 0x7f354803b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:46.366 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.725+0000 7f3560b85700 1 --2- 172.21.15.100:0/1851372948 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f355c1064a0 0x7f355c105d20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:46.366 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.725+0000 7f3560b85700 1 -- 172.21.15.100:0/1851372948 >> 172.21.15.100:0/1851372948 conn(0x7f355c0ffef0 msgr2=0x7f355c100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:46.366 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.725+0000 7f3560b85700 1 -- 172.21.15.100:0/1851372948 shutdown_connections 2024-02-14T17:39:46.366 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:45.725+0000 7f3560b85700 1 -- 172.21.15.100:0/1851372948 wait complete. 2024-02-14T17:39:46.366 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-02-14T17:39:46.366 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 5... 2024-02-14T17:39:46.484 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1851372948' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-14T17:39:46.484 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:46 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:46.199+0000 7f21ff63f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:39:46.484 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:46 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:46.310+0000 7f21ff63f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:39:46.484 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:46 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:46.405+0000 7f21ff63f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:39:46.780 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:46 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:46.501+0000 7f21ff63f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:39:46.780 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:46 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:46.680+0000 7f21ff63f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:39:47.777 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:47 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:47.645+0000 7f21ff63f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:39:48.107 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:47 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:47.775+0000 7f21ff63f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:39:48.107 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:48 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:48.105+0000 7f21ff63f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:39:48.391 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:48 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:48.195+0000 7f21ff63f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:39:48.391 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:48 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:48.389+0000 7f21ff63f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:39:48.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:48 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:48.574+0000 7f21ff63f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:39:49.119 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:48 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:48.855+0000 7f21ff63f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:39:49.119 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:49 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:49.116+0000 7f21ff63f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:39:49.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:49 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:49.352+0000 7f21ff63f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:39:50.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:50.146+0000 7f21ff63f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:39:51.025 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:50.710+0000 7f21ff63f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:39:51.025 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:50.809+0000 7f21ff63f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:39:51.026 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:39:51.023+0000 7f21ff63f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:39:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: Active manager daemon y restarted 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: Activating manager daemon y 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: osdmap e2: 0 total, 0 up, 0 in 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: mgrmap e6: y(active, starting, since 0.00335141s) 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: Manager daemon y is now available 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:51 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:52.300 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:52 smithi100 ceph-mon[30138]: Found migration_current of "None". Setting to last migration. 2024-02-14T17:39:52.300 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:52 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:52.300 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:52 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:39:52.300 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:52 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:39:52.300 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:52 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:39:52.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:52 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:39:52.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:52 smithi100 ceph-mon[30138]: mgrmap e7: y(active, since 1.00554s) 2024-02-14T17:39:52.602 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:39:52.602 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-14T17:39:52.602 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-14T17:39:52.602 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:39:52.602 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.881+0000 7f148b52a700 1 Processor -- start 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.881+0000 7f148b52a700 1 -- start start 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.881+0000 7f148b52a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f1484104700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.881+0000 7f148b52a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1484078e10 con 0x7f14841042e0 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.881+0000 7f14892c6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f1484104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.881+0000 7f14892c6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f1484104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33280/0 (socket says 172.21.15.100:33280) 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.881+0000 7f14892c6700 1 -- 172.21.15.100:0/171304294 learned_addr learned my addr 172.21.15.100:0/171304294 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.882+0000 7f14892c6700 1 -- 172.21.15.100:0/171304294 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484104c40 con 0x7f14841042e0 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.882+0000 7f14892c6700 1 --2- 172.21.15.100:0/171304294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f1484104700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f14800098a0 tx=0x7f1480009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bac1128cc86b7f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.882+0000 7f147bfff700 1 -- 172.21.15.100:0/171304294 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1480004160 con 0x7f14841042e0 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.882+0000 7f147bfff700 1 -- 172.21.15.100:0/171304294 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f14800042c0 con 0x7f14841042e0 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.882+0000 7f147bfff700 1 -- 172.21.15.100:0/171304294 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14800118a0 con 0x7f14841042e0 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.882+0000 7f148b52a700 1 -- 172.21.15.100:0/171304294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 msgr2=0x7f1484104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:52.603 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.882+0000 7f148b52a700 1 --2- 172.21.15.100:0/171304294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f1484104700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f14800098a0 tx=0x7f1480009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.883+0000 7f148b52a700 1 -- 172.21.15.100:0/171304294 shutdown_connections 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.883+0000 7f148b52a700 1 --2- 172.21.15.100:0/171304294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f1484104700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.883+0000 7f148b52a700 1 -- 172.21.15.100:0/171304294 >> 172.21.15.100:0/171304294 conn(0x7f14840ffec0 msgr2=0x7f14841022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.883+0000 7f148b52a700 1 -- 172.21.15.100:0/171304294 shutdown_connections 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.883+0000 7f148b52a700 1 -- 172.21.15.100:0/171304294 wait complete. 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.883+0000 7f148b52a700 1 Processor -- start 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.883+0000 7f148b52a700 1 -- start start 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.883+0000 7f148b52a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f148419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.883+0000 7f148b52a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148419bdd0 con 0x7f14841042e0 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.884+0000 7f14892c6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f148419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.884+0000 7f14892c6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f148419b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33292/0 (socket says 172.21.15.100:33292) 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.884+0000 7f14892c6700 1 -- 172.21.15.100:0/1610724528 learned_addr learned my addr 172.21.15.100:0/1610724528 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.884+0000 7f14892c6700 1 -- 172.21.15.100:0/1610724528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1480009550 con 0x7f14841042e0 2024-02-14T17:39:52.604 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.884+0000 7f14892c6700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f148419b890 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f14800046c0 tx=0x7f1480011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.884+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14800036a0 con 0x7f14841042e0 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.884+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f148001a430 con 0x7f14841042e0 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.884+0000 7f148b52a700 1 -- 172.21.15.100:0/1610724528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148419bfd0 con 0x7f14841042e0 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.884+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14800193c0 con 0x7f14841042e0 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.884+0000 7f148b52a700 1 -- 172.21.15.100:0/1610724528 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148419c470 con 0x7f14841042e0 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.885+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f1480003800 con 0x7f14841042e0 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.885+0000 7f147a7fc700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 0x7f147003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.885+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 --> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f147003bab0 con 0x7f1470038ee0 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.885+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f148004cf00 con 0x7f14841042e0 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.885+0000 7f1488ac5700 1 -- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 msgr2=0x7f147003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2036660321 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:46.885+0000 7f1488ac5700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 0x7f147003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:47.086+0000 7f1488ac5700 1 -- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 msgr2=0x7f147003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2036660321 2024-02-14T17:39:52.605 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:47.086+0000 7f1488ac5700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 0x7f147003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:47.487+0000 7f1488ac5700 1 -- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 msgr2=0x7f147003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2036660321 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:47.487+0000 7f1488ac5700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 0x7f147003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:48.288+0000 7f1488ac5700 1 -- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 msgr2=0x7f147003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2036660321 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:48.288+0000 7f1488ac5700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 0x7f147003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:49.889+0000 7f1488ac5700 1 -- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 msgr2=0x7f147003b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/2036660321 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:49.889+0000 7f1488ac5700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 0x7f147003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:51.039+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7f148001a930 con 0x7f14841042e0 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:51.039+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 msgr2=0x7f147003b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:51.039+0000 7f147a7fc700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 0x7f147003b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.040+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f148004e3a0 con 0x7f14841042e0 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.040+0000 7f147a7fc700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f147003c7d0 0x7f147003ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.040+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 --> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f147003bab0 con 0x7f147003c7d0 2024-02-14T17:39:52.606 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.041+0000 7f1488ac5700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f147003c7d0 0x7f147003ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.041+0000 7f1488ac5700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f147003c7d0 0x7f147003ebc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1474003a10 tx=0x7f14740092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.042+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 <== mgr.14120 v2:172.21.15.100:6800/1559623009 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f147003bab0 con 0x7f147003c7d0 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.048+0000 7f148b52a700 1 -- 172.21.15.100:0/1610724528 --> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f1484105980 con 0x7f147003c7d0 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f147a7fc700 1 -- 172.21.15.100:0/1610724528 <== mgr.14120 v2:172.21.15.100:6800/1559623009 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1484105980 con 0x7f147003c7d0 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 -- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f147003c7d0 msgr2=0x7f147003ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f147003c7d0 0x7f147003ebc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1474003a10 tx=0x7f14740092b0 comp rx=0 tx=0).stop 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 -- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 msgr2=0x7f148419b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f148419b890 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f14800046c0 tx=0x7f1480011eb0 comp rx=0 tx=0).stop 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 -- 172.21.15.100:0/1610724528 shutdown_connections 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f147003c7d0 0x7f147003ebc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f14841042e0 0x7f148419b890 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 --2- 172.21.15.100:0/1610724528 >> [v2:172.21.15.100:6800/2036660321,v1:172.21.15.100:6801/2036660321] conn(0x7f1470038ee0 0x7f147003b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 -- 172.21.15.100:0/1610724528 >> 172.21.15.100:0/1610724528 conn(0x7f14840ffec0 msgr2=0x7f1484192700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:52.607 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 -- 172.21.15.100:0/1610724528 shutdown_connections 2024-02-14T17:39:52.608 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:52.049+0000 7f148b52a700 1 -- 172.21.15.100:0/1610724528 wait complete. 2024-02-14T17:39:52.608 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 5 is available 2024-02-14T17:39:52.608 INFO:teuthology.orchestra.run.smithi100.stdout:Setting orchestrator backend to cephadm... 2024-02-14T17:39:53.271 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:53 smithi100 ceph-mon[30138]: [14/Feb/2024:17:39:51] ENGINE Bus STARTING 2024-02-14T17:39:53.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:53 smithi100 ceph-mon[30138]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-14T17:39:53.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:53 smithi100 ceph-mon[30138]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-14T17:39:53.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:53 smithi100 ceph-mon[30138]: [14/Feb/2024:17:39:52] ENGINE Serving on https://172.21.15.100:7150 2024-02-14T17:39:53.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:53 smithi100 ceph-mon[30138]: [14/Feb/2024:17:39:52] ENGINE Serving on http://172.21.15.100:8765 2024-02-14T17:39:53.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:53 smithi100 ceph-mon[30138]: [14/Feb/2024:17:39:52] ENGINE Bus STARTED 2024-02-14T17:39:53.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:53 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:53.905 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.052+0000 7f7a50b10700 1 Processor -- start 2024-02-14T17:39:53.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.052+0000 7f7a50b10700 1 -- start start 2024-02-14T17:39:53.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.052+0000 7f7a50b10700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c108910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:53.906 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.052+0000 7f7a50b10700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0797c0 con 0x7f7a4c106520 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.052+0000 7f7a4a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.052+0000 7f7a4a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33166/0 (socket says 172.21.15.100:33166) 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.052+0000 7f7a4a59c700 1 -- 172.21.15.100:0/3753954949 learned_addr learned my addr 172.21.15.100:0/3753954949 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.052+0000 7f7a4a59c700 1 -- 172.21.15.100:0/3753954949 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c108e50 con 0x7f7a4c106520 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a4a59c700 1 --2- 172.21.15.100:0/3753954949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c108910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0098a0 tx=0x7f7a3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b3b06dcf3c6268 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a4959a700 1 -- 172.21.15.100:0/3753954949 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c004160 con 0x7f7a4c106520 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a4959a700 1 -- 172.21.15.100:0/3753954949 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f7a3c0042c0 con 0x7f7a4c106520 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a4959a700 1 -- 172.21.15.100:0/3753954949 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c0118a0 con 0x7f7a4c106520 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a50b10700 1 -- 172.21.15.100:0/3753954949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 msgr2=0x7f7a4c108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a50b10700 1 --2- 172.21.15.100:0/3753954949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c108910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0098a0 tx=0x7f7a3c009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a50b10700 1 -- 172.21.15.100:0/3753954949 shutdown_connections 2024-02-14T17:39:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a50b10700 1 --2- 172.21.15.100:0/3753954949 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c108910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a50b10700 1 -- 172.21.15.100:0/3753954949 >> 172.21.15.100:0/3753954949 conn(0x7f7a4c0fff50 msgr2=0x7f7a4c102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a50b10700 1 -- 172.21.15.100:0/3753954949 shutdown_connections 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.053+0000 7f7a50b10700 1 -- 172.21.15.100:0/3753954949 wait complete. 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.054+0000 7f7a50b10700 1 Processor -- start 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.054+0000 7f7a50b10700 1 -- start start 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.054+0000 7f7a50b10700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c105da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.054+0000 7f7a50b10700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c1062e0 con 0x7f7a4c106520 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.054+0000 7f7a4a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c105da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.054+0000 7f7a4a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c105da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33170/0 (socket says 172.21.15.100:33170) 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.054+0000 7f7a4a59c700 1 -- 172.21.15.100:0/3125763977 learned_addr learned my addr 172.21.15.100:0/3125763977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.055+0000 7f7a4a59c700 1 -- 172.21.15.100:0/3125763977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c009550 con 0x7f7a4c106520 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.055+0000 7f7a4a59c700 1 --2- 172.21.15.100:0/3125763977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c105da0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c011890 tx=0x7f7a3c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.055+0000 7f7a3b7fe700 1 -- 172.21.15.100:0/3125763977 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c0036a0 con 0x7f7a4c106520 2024-02-14T17:39:53.908 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.055+0000 7f7a3b7fe700 1 -- 172.21.15.100:0/3125763977 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f7a3c01a430 con 0x7f7a4c106520 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.055+0000 7f7a3b7fe700 1 -- 172.21.15.100:0/3125763977 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c0193c0 con 0x7f7a4c106520 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.055+0000 7f7a50b10700 1 -- 172.21.15.100:0/3125763977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c1044d0 con 0x7f7a4c106520 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.055+0000 7f7a50b10700 1 -- 172.21.15.100:0/3125763977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c104970 con 0x7f7a4c106520 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.056+0000 7f7a3b7fe700 1 -- 172.21.15.100:0/3125763977 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f7a3c01a5a0 con 0x7f7a4c106520 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.056+0000 7f7a50b10700 1 -- 172.21.15.100:0/3125763977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c109b90 con 0x7f7a4c106520 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.056+0000 7f7a3b7fe700 1 --2- 172.21.15.100:0/3125763977 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f7a34038e10 0x7f7a3403b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.056+0000 7f7a3b7fe700 1 -- 172.21.15.100:0/3125763977 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7a3c04cd50 con 0x7f7a4c106520 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.056+0000 7f7a49d9b700 1 --2- 172.21.15.100:0/3125763977 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f7a34038e10 0x7f7a3403b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.056+0000 7f7a49d9b700 1 --2- 172.21.15.100:0/3125763977 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f7a34038e10 0x7f7a3403b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7a40006fd0 tx=0x7f7a40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.061+0000 7f7a3b7fe700 1 -- 172.21.15.100:0/3125763977 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a3c02d940 con 0x7f7a4c106520 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.091+0000 7f7a3b7fe700 1 -- 172.21.15.100:0/3125763977 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f7a3c01a8d0 con 0x7f7a4c106520 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.269+0000 7f7a50b10700 1 -- 172.21.15.100:0/3125763977 --> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a4c101100 con 0x7f7a34038e10 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.275+0000 7f7a3b7fe700 1 -- 172.21.15.100:0/3125763977 <== mgr.14120 v2:172.21.15.100:6800/1559623009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7a4c101100 con 0x7f7a34038e10 2024-02-14T17:39:53.909 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.277+0000 7f7a50b10700 1 -- 172.21.15.100:0/3125763977 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f7a34038e10 msgr2=0x7f7a3403b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:53.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.277+0000 7f7a50b10700 1 --2- 172.21.15.100:0/3125763977 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f7a34038e10 0x7f7a3403b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7a40006fd0 tx=0x7f7a40006e40 comp rx=0 tx=0).stop 2024-02-14T17:39:53.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.277+0000 7f7a50b10700 1 -- 172.21.15.100:0/3125763977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 msgr2=0x7f7a4c105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:53.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.277+0000 7f7a50b10700 1 --2- 172.21.15.100:0/3125763977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c105da0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c011890 tx=0x7f7a3c011970 comp rx=0 tx=0).stop 2024-02-14T17:39:53.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.277+0000 7f7a50b10700 1 -- 172.21.15.100:0/3125763977 shutdown_connections 2024-02-14T17:39:53.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.277+0000 7f7a50b10700 1 --2- 172.21.15.100:0/3125763977 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f7a34038e10 0x7f7a3403b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:53.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.277+0000 7f7a50b10700 1 --2- 172.21.15.100:0/3125763977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7a4c106520 0x7f7a4c105da0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:53.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.277+0000 7f7a50b10700 1 -- 172.21.15.100:0/3125763977 >> 172.21.15.100:0/3125763977 conn(0x7f7a4c0fff50 msgr2=0x7f7a4c1009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:53.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.277+0000 7f7a50b10700 1 -- 172.21.15.100:0/3125763977 shutdown_connections 2024-02-14T17:39:53.910 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:53.277+0000 7f7a50b10700 1 -- 172.21.15.100:0/3125763977 wait complete. 2024-02-14T17:39:54.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:54 smithi100 ceph-mon[30138]: mgrmap e8: y(active, since 2s) 2024-02-14T17:39:54.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:54 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:39:54.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:54 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:39:55.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-mon[30138]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:55.187 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-14T17:39:55.187 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.366+0000 7f6d5832e700 1 Processor -- start 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.366+0000 7f6d5832e700 1 -- start start 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.367+0000 7f6d5832e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d50104700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.367+0000 7f6d5832e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d50078e10 con 0x7f6d501042e0 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.367+0000 7f6d560ca700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d50104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.367+0000 7f6d560ca700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d50104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33174/0 (socket says 172.21.15.100:33174) 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.367+0000 7f6d560ca700 1 -- 172.21.15.100:0/3297110034 learned_addr learned my addr 172.21.15.100:0/3297110034 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.367+0000 7f6d560ca700 1 -- 172.21.15.100:0/3297110034 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d50104c40 con 0x7f6d501042e0 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.367+0000 7f6d560ca700 1 --2- 172.21.15.100:0/3297110034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d50104700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0098a0 tx=0x7f6d4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96df5ea9ea3c673 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.367+0000 7f6d550c8700 1 -- 172.21.15.100:0/3297110034 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d4c004160 con 0x7f6d501042e0 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.367+0000 7f6d550c8700 1 -- 172.21.15.100:0/3297110034 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f6d4c0042c0 con 0x7f6d501042e0 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.367+0000 7f6d550c8700 1 -- 172.21.15.100:0/3297110034 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d4c0118a0 con 0x7f6d501042e0 2024-02-14T17:39:55.188 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.368+0000 7f6d5832e700 1 -- 172.21.15.100:0/3297110034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 msgr2=0x7f6d50104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.368+0000 7f6d5832e700 1 --2- 172.21.15.100:0/3297110034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d50104700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0098a0 tx=0x7f6d4c009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.368+0000 7f6d5832e700 1 -- 172.21.15.100:0/3297110034 shutdown_connections 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.368+0000 7f6d5832e700 1 --2- 172.21.15.100:0/3297110034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d50104700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.368+0000 7f6d5832e700 1 -- 172.21.15.100:0/3297110034 >> 172.21.15.100:0/3297110034 conn(0x7f6d500ffec0 msgr2=0x7f6d501022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.368+0000 7f6d5832e700 1 -- 172.21.15.100:0/3297110034 shutdown_connections 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.368+0000 7f6d5832e700 1 -- 172.21.15.100:0/3297110034 wait complete. 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.368+0000 7f6d5832e700 1 Processor -- start 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d5832e700 1 -- start start 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d5832e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d5019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d5832e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5019bdd0 con 0x7f6d501042e0 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d560ca700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d5019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:55.189 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d560ca700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d5019b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33182/0 (socket says 172.21.15.100:33182) 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d560ca700 1 -- 172.21.15.100:0/1139509953 learned_addr learned my addr 172.21.15.100:0/1139509953 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d560ca700 1 -- 172.21.15.100:0/1139509953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c009550 con 0x7f6d501042e0 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d560ca700 1 --2- 172.21.15.100:0/1139509953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d5019b890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c003f10 tx=0x7f6d4c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d477fe700 1 -- 172.21.15.100:0/1139509953 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d4c0038b0 con 0x7f6d501042e0 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d477fe700 1 -- 172.21.15.100:0/1139509953 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f6d4c003a10 con 0x7f6d501042e0 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d5832e700 1 -- 172.21.15.100:0/1139509953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5019bfd0 con 0x7f6d501042e0 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d477fe700 1 -- 172.21.15.100:0/1139509953 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d4c0193c0 con 0x7f6d501042e0 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.369+0000 7f6d5832e700 1 -- 172.21.15.100:0/1139509953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5019c470 con 0x7f6d501042e0 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.370+0000 7f6d477fe700 1 -- 172.21.15.100:0/1139509953 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f6d4c019630 con 0x7f6d501042e0 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.370+0000 7f6d5832e700 1 -- 172.21.15.100:0/1139509953 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d501955c0 con 0x7f6d501042e0 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.370+0000 7f6d477fe700 1 --2- 172.21.15.100:0/1139509953 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f6d3c038f30 0x7f6d3c03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:55.190 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.370+0000 7f6d477fe700 1 -- 172.21.15.100:0/1139509953 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6d4c04cee0 con 0x7f6d501042e0 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.370+0000 7f6d558c9700 1 --2- 172.21.15.100:0/1139509953 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f6d3c038f30 0x7f6d3c03b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.371+0000 7f6d558c9700 1 --2- 172.21.15.100:0/1139509953 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f6d3c038f30 0x7f6d3c03b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d40006fd0 tx=0x7f6d40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.376+0000 7f6d477fe700 1 -- 172.21.15.100:0/1139509953 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6d4c02dc50 con 0x7f6d501042e0 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.568+0000 7f6d5832e700 1 -- 172.21.15.100:0/1139509953 --> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d5002d7d0 con 0x7f6d3c038f30 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.568+0000 7f6d477fe700 1 -- 172.21.15.100:0/1139509953 <== mgr.14120 v2:172.21.15.100:6800/1559623009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f6d5002d7d0 con 0x7f6d3c038f30 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.571+0000 7f6d5832e700 1 -- 172.21.15.100:0/1139509953 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f6d3c038f30 msgr2=0x7f6d3c03b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.571+0000 7f6d5832e700 1 --2- 172.21.15.100:0/1139509953 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f6d3c038f30 0x7f6d3c03b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d40006fd0 tx=0x7f6d40006e40 comp rx=0 tx=0).stop 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.571+0000 7f6d5832e700 1 -- 172.21.15.100:0/1139509953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 msgr2=0x7f6d5019b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.571+0000 7f6d5832e700 1 --2- 172.21.15.100:0/1139509953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d5019b890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c003f10 tx=0x7f6d4c0119b0 comp rx=0 tx=0).stop 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.571+0000 7f6d5832e700 1 -- 172.21.15.100:0/1139509953 shutdown_connections 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.571+0000 7f6d5832e700 1 --2- 172.21.15.100:0/1139509953 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f6d3c038f30 0x7f6d3c03b3f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.571+0000 7f6d5832e700 1 --2- 172.21.15.100:0/1139509953 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501042e0 0x7f6d5019b890 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:55.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.571+0000 7f6d5832e700 1 -- 172.21.15.100:0/1139509953 >> 172.21.15.100:0/1139509953 conn(0x7f6d500ffec0 msgr2=0x7f6d50192700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:55.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.571+0000 7f6d5832e700 1 -- 172.21.15.100:0/1139509953 shutdown_connections 2024-02-14T17:39:55.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:54.571+0000 7f6d5832e700 1 -- 172.21.15.100:0/1139509953 wait complete. 2024-02-14T17:39:55.192 INFO:teuthology.orchestra.run.smithi100.stdout:Generating ssh key... 2024-02-14T17:39:55.958 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: Generating public/private rsa key pair. 2024-02-14T17:39:55.958 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: Your identification has been saved in /tmp/tmphkfs1jgc/key. 2024-02-14T17:39:55.958 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: Your public key has been saved in /tmp/tmphkfs1jgc/key.pub. 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: The key fingerprint is: 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: SHA256:r9ceUgpdYmgVfKqvM2uCG7W9b6/athzrkfzXj3k6NkQ ceph-d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: The key's randomart image is: 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: +---[RSA 3072]----+ 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: | .o. | 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: | o. . | 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: | o oo. | 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: | . o.o E | 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: | . S.. .. | 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: | . o.+ + . | 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: | ... ..O... . | 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: | ... +==*..=oo| 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: | .. o*%O=+o+=o| 2024-02-14T17:39:55.959 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:39:55 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: +----[SHA256]-----+ 2024-02-14T17:39:56.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:56 smithi100 ceph-mon[30138]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:56.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:56 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:39:56.302 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:56 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:39:56.577 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.697+0000 7fcb16a20700 1 Processor -- start 2024-02-14T17:39:56.577 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.697+0000 7fcb16a20700 1 -- start start 2024-02-14T17:39:56.577 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.697+0000 7fcb16a20700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10107540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:56.577 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.697+0000 7fcb16a20700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb10107b10 con 0x7fcb10107120 2024-02-14T17:39:56.577 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.697+0000 7fcb0ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10107540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:56.577 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.697+0000 7fcb0ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10107540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33196/0 (socket says 172.21.15.100:33196) 2024-02-14T17:39:56.577 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.697+0000 7fcb0ffff700 1 -- 172.21.15.100:0/1257037598 learned_addr learned my addr 172.21.15.100:0/1257037598 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.698+0000 7fcb0ffff700 1 -- 172.21.15.100:0/1257037598 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb10105660 con 0x7fcb10107120 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.698+0000 7fcb0ffff700 1 --2- 172.21.15.100:0/1257037598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10107540 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcb000098a0 tx=0x7fcb00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156e5d542ba98f90 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.698+0000 7fcb0effd700 1 -- 172.21.15.100:0/1257037598 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb00004160 con 0x7fcb10107120 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.698+0000 7fcb0effd700 1 -- 172.21.15.100:0/1257037598 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fcb000042c0 con 0x7fcb10107120 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.698+0000 7fcb0effd700 1 -- 172.21.15.100:0/1257037598 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb000118a0 con 0x7fcb10107120 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.699+0000 7fcb16a20700 1 -- 172.21.15.100:0/1257037598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 msgr2=0x7fcb10107540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.699+0000 7fcb16a20700 1 --2- 172.21.15.100:0/1257037598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10107540 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcb000098a0 tx=0x7fcb00009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.699+0000 7fcb16a20700 1 -- 172.21.15.100:0/1257037598 shutdown_connections 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.699+0000 7fcb16a20700 1 --2- 172.21.15.100:0/1257037598 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10107540 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.699+0000 7fcb16a20700 1 -- 172.21.15.100:0/1257037598 >> 172.21.15.100:0/1257037598 conn(0x7fcb10101000 msgr2=0x7fcb10103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.699+0000 7fcb16a20700 1 -- 172.21.15.100:0/1257037598 shutdown_connections 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.699+0000 7fcb16a20700 1 -- 172.21.15.100:0/1257037598 wait complete. 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.699+0000 7fcb16a20700 1 Processor -- start 2024-02-14T17:39:56.578 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.699+0000 7fcb16a20700 1 -- start start 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb16a20700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb16a20700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb10199cb0 con 0x7fcb10107120 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb0ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb0ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33202/0 (socket says 172.21.15.100:33202) 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb0ffff700 1 -- 172.21.15.100:0/3729857003 learned_addr learned my addr 172.21.15.100:0/3729857003 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb0ffff700 1 -- 172.21.15.100:0/3729857003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00009550 con 0x7fcb10107120 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb0ffff700 1 --2- 172.21.15.100:0/3729857003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10199770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb00003f10 tx=0x7fcb00011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb0d7fa700 1 -- 172.21.15.100:0/3729857003 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb000036a0 con 0x7fcb10107120 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb0d7fa700 1 -- 172.21.15.100:0/3729857003 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fcb0001a430 con 0x7fcb10107120 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb16a20700 1 -- 172.21.15.100:0/3729857003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb10199eb0 con 0x7fcb10107120 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb0d7fa700 1 -- 172.21.15.100:0/3729857003 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb00024410 con 0x7fcb10107120 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.700+0000 7fcb16a20700 1 -- 172.21.15.100:0/3729857003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1019a350 con 0x7fcb10107120 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.701+0000 7fcb0d7fa700 1 -- 172.21.15.100:0/3729857003 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fcb0002d3f0 con 0x7fcb10107120 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.701+0000 7fcb16a20700 1 -- 172.21.15.100:0/3729857003 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb100632a0 con 0x7fcb10107120 2024-02-14T17:39:56.579 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.701+0000 7fcb0d7fa700 1 --2- 172.21.15.100:0/3729857003 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fcaf8038ee0 0x7fcaf803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.701+0000 7fcb0d7fa700 1 -- 172.21.15.100:0/3729857003 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcb0004ccf0 con 0x7fcb10107120 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.701+0000 7fcb0f7fe700 1 --2- 172.21.15.100:0/3729857003 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fcaf8038ee0 0x7fcaf803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.702+0000 7fcb0f7fe700 1 --2- 172.21.15.100:0/3729857003 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fcaf8038ee0 0x7fcaf803b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800ad30 tx=0x7fcb080093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.706+0000 7fcb0d7fa700 1 -- 172.21.15.100:0/3729857003 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb00051070 con 0x7fcb10107120 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.899+0000 7fcb16a20700 1 -- 172.21.15.100:0/3729857003 --> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb10102220 con 0x7fcaf8038ee0 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.955+0000 7fcb0d7fa700 1 -- 172.21.15.100:0/3729857003 <== mgr.14120 v2:172.21.15.100:6800/1559623009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcb10102220 con 0x7fcaf8038ee0 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.958+0000 7fcb16a20700 1 -- 172.21.15.100:0/3729857003 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fcaf8038ee0 msgr2=0x7fcaf803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.958+0000 7fcb16a20700 1 --2- 172.21.15.100:0/3729857003 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fcaf8038ee0 0x7fcaf803b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800ad30 tx=0x7fcb080093f0 comp rx=0 tx=0).stop 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.958+0000 7fcb16a20700 1 -- 172.21.15.100:0/3729857003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 msgr2=0x7fcb10199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.958+0000 7fcb16a20700 1 --2- 172.21.15.100:0/3729857003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10199770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb00003f10 tx=0x7fcb00011940 comp rx=0 tx=0).stop 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.958+0000 7fcb16a20700 1 -- 172.21.15.100:0/3729857003 shutdown_connections 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.958+0000 7fcb16a20700 1 --2- 172.21.15.100:0/3729857003 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fcaf8038ee0 0x7fcaf803b3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:56.580 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.958+0000 7fcb16a20700 1 --2- 172.21.15.100:0/3729857003 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb10107120 0x7fcb10199770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:56.581 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.958+0000 7fcb16a20700 1 -- 172.21.15.100:0/3729857003 >> 172.21.15.100:0/3729857003 conn(0x7fcb10101000 msgr2=0x7fcb10101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:56.581 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.958+0000 7fcb16a20700 1 -- 172.21.15.100:0/3729857003 shutdown_connections 2024-02-14T17:39:56.581 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:55.958+0000 7fcb16a20700 1 -- 172.21.15.100:0/3729857003 wait complete. 2024-02-14T17:39:57.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:57 smithi100 ceph-mon[30138]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:57.227 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:57 smithi100 ceph-mon[30138]: Generating ssh key... 2024-02-14T17:39:57.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmZFIESaVkU2Xvrkbznvx6UNUzPJCbAqX4naRSa7PcLSYoY7JMGugN7P6niRwe2gzkwxL1rP8S48/fN7QzSKqNFqBSG5Yc2ADny7NxcGtpgyM4Sg+qkQWdRMuZF4Bkq8ooXXMr024ogmw7i/k8vGTTpmaXlHU8F27x9+By3EBtheMX4bUxsiilIW8qOt1N7sCHOd207eoakdfHSTetrwTYvM2BSzeSP42tRGaZieHpU+urInLy+oELI7SdDeV4Z712iSglhtdkKM+ycd/J9o5IkMFQkuVVBB6G/ErtEo93MWKGseIWgQt4NYvjWTEPTJTmC5va1GMVMmAUCFHMBW6qNmr23ksfFyi5LYThyyKSeVHGyqW7V+ZFEG9aWQTzP4rotTXg+xvNf29cl18H9W9HYXMUPZ+et9b8iwSmZ7TGymtoqzfWkdEskqHBQhDtbqGGlk1gfPjjarSB3j0tRalOfsagyWS7hXOFI6ei51OsfVE62gWdHk1GxOfb+xN0RJ8= ceph-d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:39:57.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.020+0000 7ff96530b700 1 Processor -- start 2024-02-14T17:39:57.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.020+0000 7ff96530b700 1 -- start start 2024-02-14T17:39:57.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.020+0000 7ff96530b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960104d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:57.824 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.020+0000 7ff96530b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9601052f0 con 0x7ff960104900 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.021+0000 7ff95effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960104d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.021+0000 7ff95effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960104d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33218/0 (socket says 172.21.15.100:33218) 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.021+0000 7ff95effd700 1 -- 172.21.15.100:0/1966405879 learned_addr learned my addr 172.21.15.100:0/1966405879 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.021+0000 7ff95effd700 1 -- 172.21.15.100:0/1966405879 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960105b00 con 0x7ff960104900 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.021+0000 7ff95effd700 1 --2- 172.21.15.100:0/1966405879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960104d20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff9580098a0 tx=0x7ff958009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ebbb9fab288d72 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.021+0000 7ff95dffb700 1 -- 172.21.15.100:0/1966405879 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff958004160 con 0x7ff960104900 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.021+0000 7ff95dffb700 1 -- 172.21.15.100:0/1966405879 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7ff9580042c0 con 0x7ff960104900 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.021+0000 7ff95dffb700 1 -- 172.21.15.100:0/1966405879 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9580118a0 con 0x7ff960104900 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.022+0000 7ff96530b700 1 -- 172.21.15.100:0/1966405879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 msgr2=0x7ff960104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.022+0000 7ff96530b700 1 --2- 172.21.15.100:0/1966405879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960104d20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff9580098a0 tx=0x7ff958009bb0 comp rx=0 tx=0).stop 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.022+0000 7ff96530b700 1 -- 172.21.15.100:0/1966405879 shutdown_connections 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.022+0000 7ff96530b700 1 --2- 172.21.15.100:0/1966405879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960104d20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.022+0000 7ff96530b700 1 -- 172.21.15.100:0/1966405879 >> 172.21.15.100:0/1966405879 conn(0x7ff9600ffec0 msgr2=0x7ff9601022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:57.825 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.022+0000 7ff96530b700 1 -- 172.21.15.100:0/1966405879 shutdown_connections 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.022+0000 7ff96530b700 1 -- 172.21.15.100:0/1966405879 wait complete. 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.022+0000 7ff96530b700 1 Processor -- start 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff96530b700 1 -- start start 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff96530b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff96530b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff960198b40 con 0x7ff960104900 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff95effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff95effd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33228/0 (socket says 172.21.15.100:33228) 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff95effd700 1 -- 172.21.15.100:0/3877013316 learned_addr learned my addr 172.21.15.100:0/3877013316 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff95effd700 1 -- 172.21.15.100:0/3877013316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958009550 con 0x7ff960104900 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff95effd700 1 --2- 172.21.15.100:0/3877013316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960198600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff958006af0 tx=0x7ff9580119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff947fff700 1 -- 172.21.15.100:0/3877013316 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9580036a0 con 0x7ff960104900 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff947fff700 1 -- 172.21.15.100:0/3877013316 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7ff95801a430 con 0x7ff960104900 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff96530b700 1 -- 172.21.15.100:0/3877013316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff960198d40 con 0x7ff960104900 2024-02-14T17:39:57.826 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff947fff700 1 -- 172.21.15.100:0/3877013316 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9580193c0 con 0x7ff960104900 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.023+0000 7ff96530b700 1 -- 172.21.15.100:0/3877013316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9601991e0 con 0x7ff960104900 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.024+0000 7ff947fff700 1 -- 172.21.15.100:0/3877013316 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7ff958019520 con 0x7ff960104900 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.024+0000 7ff96530b700 1 -- 172.21.15.100:0/3877013316 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9600632a0 con 0x7ff960104900 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.024+0000 7ff947fff700 1 --2- 172.21.15.100:0/3877013316 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7ff948038f30 0x7ff94803b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.024+0000 7ff947fff700 1 -- 172.21.15.100:0/3877013316 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff95804cef0 con 0x7ff960104900 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.025+0000 7ff95e7fc700 1 --2- 172.21.15.100:0/3877013316 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7ff948038f30 0x7ff94803b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.025+0000 7ff95e7fc700 1 --2- 172.21.15.100:0/3877013316 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7ff948038f30 0x7ff94803b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff95400ad30 tx=0x7ff9540093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.029+0000 7ff947fff700 1 -- 172.21.15.100:0/3877013316 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff958010e80 con 0x7ff960104900 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.225+0000 7ff96530b700 1 -- 172.21.15.100:0/3877013316 --> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9601011c0 con 0x7ff948038f30 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.226+0000 7ff947fff700 1 -- 172.21.15.100:0/3877013316 <== mgr.14120 v2:172.21.15.100:6800/1559623009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7ff9601011c0 con 0x7ff948038f30 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.228+0000 7ff96530b700 1 -- 172.21.15.100:0/3877013316 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7ff948038f30 msgr2=0x7ff94803b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.228+0000 7ff96530b700 1 --2- 172.21.15.100:0/3877013316 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7ff948038f30 0x7ff94803b3f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff95400ad30 tx=0x7ff9540093f0 comp rx=0 tx=0).stop 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.228+0000 7ff96530b700 1 -- 172.21.15.100:0/3877013316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 msgr2=0x7ff960198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:39:57.827 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.228+0000 7ff96530b700 1 --2- 172.21.15.100:0/3877013316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960198600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff958006af0 tx=0x7ff9580119b0 comp rx=0 tx=0).stop 2024-02-14T17:39:57.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.228+0000 7ff96530b700 1 -- 172.21.15.100:0/3877013316 shutdown_connections 2024-02-14T17:39:57.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.228+0000 7ff96530b700 1 --2- 172.21.15.100:0/3877013316 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7ff948038f30 0x7ff94803b3f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:57.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.228+0000 7ff96530b700 1 --2- 172.21.15.100:0/3877013316 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff960104900 0x7ff960198600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:39:57.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.228+0000 7ff96530b700 1 -- 172.21.15.100:0/3877013316 >> 172.21.15.100:0/3877013316 conn(0x7ff9600ffec0 msgr2=0x7ff960100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:39:57.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.228+0000 7ff96530b700 1 -- 172.21.15.100:0/3877013316 shutdown_connections 2024-02-14T17:39:57.828 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:57.228+0000 7ff96530b700 1 -- 172.21.15.100:0/3877013316 wait complete. 2024-02-14T17:39:57.828 INFO:teuthology.orchestra.run.smithi100.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-14T17:39:57.828 INFO:teuthology.orchestra.run.smithi100.stdout:Adding key to root@localhost authorized_keys... 2024-02-14T17:39:57.828 INFO:teuthology.orchestra.run.smithi100.stdout:Adding host smithi100... 2024-02-14T17:39:58.332 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:58 smithi100 ceph-mon[30138]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:59.546 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:59 smithi100 ceph-mon[30138]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "addr": "172.21.15.100", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:39:59.547 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:39:59 smithi100 ceph-mon[30138]: Deploying cephadm binary to smithi100 2024-02-14T17:40:03.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Added host 'smithi100' with addr '172.21.15.100' 2024-02-14T17:40:03.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.329+0000 7feafe9bc700 1 Processor -- start 2024-02-14T17:40:03.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.329+0000 7feafe9bc700 1 -- start start 2024-02-14T17:40:03.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.329+0000 7feafe9bc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8107550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:03.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.329+0000 7feafe9bc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8107b20 con 0x7feaf8107130 2024-02-14T17:40:03.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.329+0000 7feaf7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:03.432 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.329+0000 7feaf7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8107550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33238/0 (socket says 172.21.15.100:33238) 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.329+0000 7feaf7fff700 1 -- 172.21.15.100:0/1685400172 learned_addr learned my addr 172.21.15.100:0/1685400172 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.330+0000 7feaf7fff700 1 -- 172.21.15.100:0/1685400172 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8105670 con 0x7feaf8107130 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.330+0000 7feaf7fff700 1 --2- 172.21.15.100:0/1685400172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8107550 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feae80098a0 tx=0x7feae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd60b29f395f16a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.330+0000 7feaf6ffd700 1 -- 172.21.15.100:0/1685400172 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feae8004160 con 0x7feaf8107130 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.330+0000 7feaf6ffd700 1 -- 172.21.15.100:0/1685400172 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7feae80042c0 con 0x7feaf8107130 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.330+0000 7feaf6ffd700 1 -- 172.21.15.100:0/1685400172 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feae80118a0 con 0x7feaf8107130 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.330+0000 7feafe9bc700 1 -- 172.21.15.100:0/1685400172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 msgr2=0x7feaf8107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.330+0000 7feafe9bc700 1 --2- 172.21.15.100:0/1685400172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8107550 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feae80098a0 tx=0x7feae8009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.330+0000 7feafe9bc700 1 -- 172.21.15.100:0/1685400172 shutdown_connections 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.330+0000 7feafe9bc700 1 --2- 172.21.15.100:0/1685400172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8107550 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.330+0000 7feafe9bc700 1 -- 172.21.15.100:0/1685400172 >> 172.21.15.100:0/1685400172 conn(0x7feaf8101030 msgr2=0x7feaf8103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.331+0000 7feafe9bc700 1 -- 172.21.15.100:0/1685400172 shutdown_connections 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.331+0000 7feafe9bc700 1 -- 172.21.15.100:0/1685400172 wait complete. 2024-02-14T17:40:03.433 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.331+0000 7feafe9bc700 1 Processor -- start 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.331+0000 7feafe9bc700 1 -- start start 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.332+0000 7feafe9bc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.332+0000 7feafe9bc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8199c80 con 0x7feaf8107130 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.332+0000 7feaf7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.332+0000 7feaf7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33254/0 (socket says 172.21.15.100:33254) 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.332+0000 7feaf7fff700 1 -- 172.21.15.100:0/3082865672 learned_addr learned my addr 172.21.15.100:0/3082865672 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.332+0000 7feaf7fff700 1 -- 172.21.15.100:0/3082865672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8009550 con 0x7feaf8107130 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.332+0000 7feaf7fff700 1 --2- 172.21.15.100:0/3082865672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8199740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feae8003f10 tx=0x7feae8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.332+0000 7feaf57fa700 1 -- 172.21.15.100:0/3082865672 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feae80036a0 con 0x7feaf8107130 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.332+0000 7feafe9bc700 1 -- 172.21.15.100:0/3082865672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf8199e80 con 0x7feaf8107130 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.333+0000 7feaf57fa700 1 -- 172.21.15.100:0/3082865672 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7feae801a430 con 0x7feaf8107130 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.333+0000 7feafe9bc700 1 -- 172.21.15.100:0/3082865672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf819a320 con 0x7feaf8107130 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.333+0000 7feaf57fa700 1 -- 172.21.15.100:0/3082865672 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feae8024410 con 0x7feaf8107130 2024-02-14T17:40:03.434 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.333+0000 7feafe9bc700 1 -- 172.21.15.100:0/3082865672 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae4005150 con 0x7feaf8107130 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.334+0000 7feaf57fa700 1 -- 172.21.15.100:0/3082865672 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7feae8003800 con 0x7feaf8107130 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.334+0000 7feaf57fa700 1 --2- 172.21.15.100:0/3082865672 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7feae0038f30 0x7feae003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.334+0000 7feaf57fa700 1 -- 172.21.15.100:0/3082865672 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7feae804cc90 con 0x7feaf8107130 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.334+0000 7feaf77fe700 1 --2- 172.21.15.100:0/3082865672 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7feae0038f30 0x7feae003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.334+0000 7feaf77fe700 1 --2- 172.21.15.100:0/3082865672 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7feae0038f30 0x7feae003b3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feaec00ad30 tx=0x7feaec0093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.339+0000 7feaf57fa700 1 -- 172.21.15.100:0/3082865672 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feae8018530 con 0x7feaf8107130 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:39:58.534+0000 7feafe9bc700 1 -- 172.21.15.100:0/3082865672 --> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi100", "addr": "172.21.15.100", "target": ["mon-mgr", ""]}) v1 -- 0x7feae4000bf0 con 0x7feae0038f30 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.802+0000 7feaf57fa700 1 -- 172.21.15.100:0/3082865672 <== mgr.14120 v2:172.21.15.100:6800/1559623009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7feae4000bf0 con 0x7feae0038f30 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.804+0000 7feafe9bc700 1 -- 172.21.15.100:0/3082865672 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7feae0038f30 msgr2=0x7feae003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.804+0000 7feafe9bc700 1 --2- 172.21.15.100:0/3082865672 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7feae0038f30 0x7feae003b3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feaec00ad30 tx=0x7feaec0093f0 comp rx=0 tx=0).stop 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.804+0000 7feafe9bc700 1 -- 172.21.15.100:0/3082865672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 msgr2=0x7feaf8199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.804+0000 7feafe9bc700 1 --2- 172.21.15.100:0/3082865672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8199740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feae8003f10 tx=0x7feae8011940 comp rx=0 tx=0).stop 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.805+0000 7feafe9bc700 1 -- 172.21.15.100:0/3082865672 shutdown_connections 2024-02-14T17:40:03.435 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.805+0000 7feafe9bc700 1 --2- 172.21.15.100:0/3082865672 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7feae0038f30 0x7feae003b3f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:03.436 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.805+0000 7feafe9bc700 1 --2- 172.21.15.100:0/3082865672 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feaf8107130 0x7feaf8199740 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:03.436 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.805+0000 7feafe9bc700 1 -- 172.21.15.100:0/3082865672 >> 172.21.15.100:0/3082865672 conn(0x7feaf8101030 msgr2=0x7feaf8101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:03.436 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.805+0000 7feafe9bc700 1 -- 172.21.15.100:0/3082865672 shutdown_connections 2024-02-14T17:40:03.436 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:02.805+0000 7feafe9bc700 1 -- 172.21.15.100:0/3082865672 wait complete. 2024-02-14T17:40:03.436 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mon service... 2024-02-14T17:40:04.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:03 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:40:04.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:03 smithi100 ceph-mon[30138]: Added host smithi100 2024-02-14T17:40:04.131 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:03 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:04.773 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-14T17:40:04.773 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.914+0000 7f0b36751700 1 Processor -- start 2024-02-14T17:40:04.773 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.914+0000 7f0b36751700 1 -- start start 2024-02-14T17:40:04.773 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.914+0000 7f0b36751700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:04.773 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.914+0000 7f0b36751700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30079720 con 0x7f0b301064a0 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.914+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.914+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35204/0 (socket says 172.21.15.100:35204) 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.914+0000 7f0b2ffff700 1 -- 172.21.15.100:0/1268742874 learned_addr learned my addr 172.21.15.100:0/1268742874 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.914+0000 7f0b2ffff700 1 -- 172.21.15.100:0/1268742874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b30108dd0 con 0x7f0b301064a0 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.914+0000 7f0b2ffff700 1 --2- 172.21.15.100:0/1268742874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30108890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b200098a0 tx=0x7f0b20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a19a527caf8eaf server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.915+0000 7f0b2effd700 1 -- 172.21.15.100:0/1268742874 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b20004160 con 0x7f0b301064a0 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.915+0000 7f0b2effd700 1 -- 172.21.15.100:0/1268742874 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f0b200042c0 con 0x7f0b301064a0 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.915+0000 7f0b2effd700 1 -- 172.21.15.100:0/1268742874 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b200118a0 con 0x7f0b301064a0 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.915+0000 7f0b36751700 1 -- 172.21.15.100:0/1268742874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 msgr2=0x7f0b30108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.915+0000 7f0b36751700 1 --2- 172.21.15.100:0/1268742874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30108890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b200098a0 tx=0x7f0b20009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.915+0000 7f0b36751700 1 -- 172.21.15.100:0/1268742874 shutdown_connections 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.915+0000 7f0b36751700 1 --2- 172.21.15.100:0/1268742874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30108890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.915+0000 7f0b36751700 1 -- 172.21.15.100:0/1268742874 >> 172.21.15.100:0/1268742874 conn(0x7f0b300ffef0 msgr2=0x7f0b30102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:04.774 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.915+0000 7f0b36751700 1 -- 172.21.15.100:0/1268742874 shutdown_connections 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.915+0000 7f0b36751700 1 -- 172.21.15.100:0/1268742874 wait complete. 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.916+0000 7f0b36751700 1 Processor -- start 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.916+0000 7f0b36751700 1 -- start start 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.916+0000 7f0b36751700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.916+0000 7f0b36751700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30199ba0 con 0x7f0b301064a0 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.916+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.916+0000 7f0b2ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35208/0 (socket says 172.21.15.100:35208) 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.916+0000 7f0b2ffff700 1 -- 172.21.15.100:0/1197846018 learned_addr learned my addr 172.21.15.100:0/1197846018 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.916+0000 7f0b2ffff700 1 -- 172.21.15.100:0/1197846018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b20009550 con 0x7f0b301064a0 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.916+0000 7f0b2ffff700 1 --2- 172.21.15.100:0/1197846018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30199660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b20011890 tx=0x7f0b20011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.917+0000 7f0b2d7fa700 1 -- 172.21.15.100:0/1197846018 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b200036a0 con 0x7f0b301064a0 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.917+0000 7f0b36751700 1 -- 172.21.15.100:0/1197846018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b30199da0 con 0x7f0b301064a0 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.917+0000 7f0b2d7fa700 1 -- 172.21.15.100:0/1197846018 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f0b2001a430 con 0x7f0b301064a0 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.917+0000 7f0b2d7fa700 1 -- 172.21.15.100:0/1197846018 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b200193c0 con 0x7f0b301064a0 2024-02-14T17:40:04.775 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.917+0000 7f0b36751700 1 -- 172.21.15.100:0/1197846018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3019a240 con 0x7f0b301064a0 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.918+0000 7f0b2d7fa700 1 -- 172.21.15.100:0/1197846018 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f0b20019640 con 0x7f0b301064a0 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.918+0000 7f0b36751700 1 -- 172.21.15.100:0/1197846018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b30109aa0 con 0x7f0b301064a0 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.918+0000 7f0b2d7fa700 1 --2- 172.21.15.100:0/1197846018 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0b18038ee0 0x7f0b1803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.918+0000 7f0b2d7fa700 1 -- 172.21.15.100:0/1197846018 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0b2004d0d0 con 0x7f0b301064a0 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.918+0000 7f0b2f7fe700 1 --2- 172.21.15.100:0/1197846018 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0b18038ee0 0x7f0b1803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.918+0000 7f0b2f7fe700 1 --2- 172.21.15.100:0/1197846018 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0b18038ee0 0x7f0b1803b3a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b24006fd0 tx=0x7f0b24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:03.922+0000 7f0b2d7fa700 1 -- 172.21.15.100:0/1197846018 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0b20023bc0 con 0x7f0b301064a0 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.129+0000 7f0b36751700 1 -- 172.21.15.100:0/1197846018 --> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0b30101140 con 0x7f0b18038ee0 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.136+0000 7f0b2d7fa700 1 -- 172.21.15.100:0/1197846018 <== mgr.14120 v2:172.21.15.100:6800/1559623009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0b30101140 con 0x7f0b18038ee0 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.138+0000 7f0b36751700 1 -- 172.21.15.100:0/1197846018 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0b18038ee0 msgr2=0x7f0b1803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.138+0000 7f0b36751700 1 --2- 172.21.15.100:0/1197846018 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0b18038ee0 0x7f0b1803b3a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b24006fd0 tx=0x7f0b24006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.138+0000 7f0b36751700 1 -- 172.21.15.100:0/1197846018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 msgr2=0x7f0b30199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:04.776 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.138+0000 7f0b36751700 1 --2- 172.21.15.100:0/1197846018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30199660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b20011890 tx=0x7f0b20011970 comp rx=0 tx=0).stop 2024-02-14T17:40:04.777 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.138+0000 7f0b36751700 1 -- 172.21.15.100:0/1197846018 shutdown_connections 2024-02-14T17:40:04.777 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.138+0000 7f0b36751700 1 --2- 172.21.15.100:0/1197846018 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0b18038ee0 0x7f0b1803b3a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:04.777 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.138+0000 7f0b36751700 1 --2- 172.21.15.100:0/1197846018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b301064a0 0x7f0b30199660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:04.777 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.138+0000 7f0b36751700 1 -- 172.21.15.100:0/1197846018 >> 172.21.15.100:0/1197846018 conn(0x7f0b300ffef0 msgr2=0x7f0b30100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:04.777 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.138+0000 7f0b36751700 1 -- 172.21.15.100:0/1197846018 shutdown_connections 2024-02-14T17:40:04.777 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:04.138+0000 7f0b36751700 1 -- 172.21.15.100:0/1197846018 wait complete. 2024-02-14T17:40:04.777 INFO:teuthology.orchestra.run.smithi100.stdout:Deploying unmanaged mgr service... 2024-02-14T17:40:05.353 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:05 smithi100 ceph-mon[30138]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:05.353 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:05 smithi100 ceph-mon[30138]: Saving service mon spec with placement count:5 2024-02-14T17:40:05.353 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:05 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:40:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:06 smithi100 ceph-mon[30138]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:06 smithi100 ceph-mon[30138]: Saving service mgr spec with placement count:2 2024-02-14T17:40:06.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:06 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:40:07.028 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-14T17:40:07.028 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.243+0000 7fa53f418700 1 Processor -- start 2024-02-14T17:40:07.028 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.243+0000 7fa53f418700 1 -- start start 2024-02-14T17:40:07.028 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.243+0000 7fa53f418700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa5381088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:07.028 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.243+0000 7fa53f418700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380796c0 con 0x7fa5381064c0 2024-02-14T17:40:07.028 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.244+0000 7fa53d1b4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa5381088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.244+0000 7fa53d1b4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa5381088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35224/0 (socket says 172.21.15.100:35224) 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.244+0000 7fa53d1b4700 1 -- 172.21.15.100:0/2693789684 learned_addr learned my addr 172.21.15.100:0/2693789684 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.244+0000 7fa53d1b4700 1 -- 172.21.15.100:0/2693789684 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa538108df0 con 0x7fa5381064c0 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.244+0000 7fa53d1b4700 1 --2- 172.21.15.100:0/2693789684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa5381088b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa530009e80 tx=0x7fa53000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8761d84b33f93d17 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.244+0000 7fa537fff700 1 -- 172.21.15.100:0/2693789684 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa53000e070 con 0x7fa5381064c0 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.244+0000 7fa537fff700 1 -- 172.21.15.100:0/2693789684 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fa530003e80 con 0x7fa5381064c0 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.244+0000 7fa537fff700 1 -- 172.21.15.100:0/2693789684 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa530004370 con 0x7fa5381064c0 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.245+0000 7fa53f418700 1 -- 172.21.15.100:0/2693789684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 msgr2=0x7fa5381088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.245+0000 7fa53f418700 1 --2- 172.21.15.100:0/2693789684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa5381088b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa530009e80 tx=0x7fa53000b330 comp rx=0 tx=0).stop 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.245+0000 7fa53f418700 1 -- 172.21.15.100:0/2693789684 shutdown_connections 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.245+0000 7fa53f418700 1 --2- 172.21.15.100:0/2693789684 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa5381088b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:07.029 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.245+0000 7fa53f418700 1 -- 172.21.15.100:0/2693789684 >> 172.21.15.100:0/2693789684 conn(0x7fa5380ffe50 msgr2=0x7fa5381022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.245+0000 7fa53f418700 1 -- 172.21.15.100:0/2693789684 shutdown_connections 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.245+0000 7fa53f418700 1 -- 172.21.15.100:0/2693789684 wait complete. 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.245+0000 7fa53f418700 1 Processor -- start 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.245+0000 7fa53f418700 1 -- start start 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa53f418700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa538199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa53f418700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa538199c70 con 0x7fa5381064c0 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa53d1b4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa538199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa53d1b4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa538199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35240/0 (socket says 172.21.15.100:35240) 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa53d1b4700 1 -- 172.21.15.100:0/3946346483 learned_addr learned my addr 172.21.15.100:0/3946346483 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa53d1b4700 1 -- 172.21.15.100:0/3946346483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa530009b30 con 0x7fa5381064c0 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa53d1b4700 1 --2- 172.21.15.100:0/3946346483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa538199730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa530004000 tx=0x7fa5300048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa5367fc700 1 -- 172.21.15.100:0/3946346483 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa53000e040 con 0x7fa5381064c0 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa5367fc700 1 -- 172.21.15.100:0/3946346483 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fa53000b8c0 con 0x7fa5381064c0 2024-02-14T17:40:07.030 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa53f418700 1 -- 172.21.15.100:0/3946346483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa538199e70 con 0x7fa5381064c0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa5367fc700 1 -- 172.21.15.100:0/3946346483 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa530012770 con 0x7fa5381064c0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.246+0000 7fa53f418700 1 -- 172.21.15.100:0/3946346483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53819a370 con 0x7fa5381064c0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.247+0000 7fa5367fc700 1 -- 172.21.15.100:0/3946346483 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fa5300129f0 con 0x7fa5381064c0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.247+0000 7fa53f418700 1 -- 172.21.15.100:0/3946346483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5380632a0 con 0x7fa5381064c0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.247+0000 7fa5367fc700 1 --2- 172.21.15.100:0/3946346483 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa520038ee0 0x7fa52003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.247+0000 7fa5367fc700 1 -- 172.21.15.100:0/3946346483 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa53004d0c0 con 0x7fa5381064c0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.247+0000 7fa53c9b3700 1 --2- 172.21.15.100:0/3946346483 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa520038ee0 0x7fa52003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.248+0000 7fa53c9b3700 1 --2- 172.21.15.100:0/3946346483 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa520038ee0 0x7fa52003b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa52c006fd0 tx=0x7fa52c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.252+0000 7fa5367fc700 1 -- 172.21.15.100:0/3946346483 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa530025780 con 0x7fa5381064c0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.467+0000 7fa53f418700 1 -- 172.21.15.100:0/3946346483 --> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa538101070 con 0x7fa520038ee0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.471+0000 7fa5367fc700 1 -- 172.21.15.100:0/3946346483 <== mgr.14120 v2:172.21.15.100:6800/1559623009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa538101070 con 0x7fa520038ee0 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.473+0000 7fa53f418700 1 -- 172.21.15.100:0/3946346483 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa520038ee0 msgr2=0x7fa52003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:07.031 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.473+0000 7fa53f418700 1 --2- 172.21.15.100:0/3946346483 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa520038ee0 0x7fa52003b3a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa52c006fd0 tx=0x7fa52c006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:07.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.473+0000 7fa53f418700 1 -- 172.21.15.100:0/3946346483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 msgr2=0x7fa538199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:07.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.473+0000 7fa53f418700 1 --2- 172.21.15.100:0/3946346483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa538199730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa530004000 tx=0x7fa5300048d0 comp rx=0 tx=0).stop 2024-02-14T17:40:07.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.473+0000 7fa53f418700 1 -- 172.21.15.100:0/3946346483 shutdown_connections 2024-02-14T17:40:07.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.473+0000 7fa53f418700 1 --2- 172.21.15.100:0/3946346483 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa520038ee0 0x7fa52003b3a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:07.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.473+0000 7fa53f418700 1 --2- 172.21.15.100:0/3946346483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa5381064c0 0x7fa538199730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:07.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.473+0000 7fa53f418700 1 -- 172.21.15.100:0/3946346483 >> 172.21.15.100:0/3946346483 conn(0x7fa5380ffe50 msgr2=0x7fa538100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:07.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.473+0000 7fa53f418700 1 -- 172.21.15.100:0/3946346483 shutdown_connections 2024-02-14T17:40:07.032 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:05.473+0000 7fa53f418700 1 -- 172.21.15.100:0/3946346483 wait complete. 2024-02-14T17:40:08.050 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:07 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:40:08.050 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:07 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:40:08.384 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.521+0000 7f49dbc7d700 1 Processor -- start 2024-02-14T17:40:08.384 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.522+0000 7f49dbc7d700 1 -- start start 2024-02-14T17:40:08.384 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.522+0000 7f49dbc7d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4108860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.522+0000 7f49dbc7d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d4078e90 con 0x7f49d4106470 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.522+0000 7f49d9a19700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4108860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.522+0000 7f49d9a19700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4108860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35252/0 (socket says 172.21.15.100:35252) 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.522+0000 7f49d9a19700 1 -- 172.21.15.100:0/1419897487 learned_addr learned my addr 172.21.15.100:0/1419897487 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.522+0000 7f49d9a19700 1 -- 172.21.15.100:0/1419897487 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4108da0 con 0x7f49d4106470 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.522+0000 7f49d9a19700 1 --2- 172.21.15.100:0/1419897487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4108860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49d00098a0 tx=0x7f49d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d5eb18bd278d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.523+0000 7f49d8a17700 1 -- 172.21.15.100:0/1419897487 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d0004160 con 0x7f49d4106470 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.523+0000 7f49d8a17700 1 -- 172.21.15.100:0/1419897487 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f49d00042c0 con 0x7f49d4106470 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.523+0000 7f49d8a17700 1 -- 172.21.15.100:0/1419897487 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d0011850 con 0x7f49d4106470 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.523+0000 7f49dbc7d700 1 -- 172.21.15.100:0/1419897487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 msgr2=0x7f49d4108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.523+0000 7f49dbc7d700 1 --2- 172.21.15.100:0/1419897487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4108860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49d00098a0 tx=0x7f49d0009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:08.385 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.523+0000 7f49dbc7d700 1 -- 172.21.15.100:0/1419897487 shutdown_connections 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.523+0000 7f49dbc7d700 1 --2- 172.21.15.100:0/1419897487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4108860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.523+0000 7f49dbc7d700 1 -- 172.21.15.100:0/1419897487 >> 172.21.15.100:0/1419897487 conn(0x7f49d40ffec0 msgr2=0x7f49d41022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.523+0000 7f49dbc7d700 1 -- 172.21.15.100:0/1419897487 shutdown_connections 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.523+0000 7f49dbc7d700 1 -- 172.21.15.100:0/1419897487 wait complete. 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.524+0000 7f49dbc7d700 1 Processor -- start 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.524+0000 7f49dbc7d700 1 -- start start 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.524+0000 7f49dbc7d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4105cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.524+0000 7f49dbc7d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d4106230 con 0x7f49d4106470 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.524+0000 7f49d9a19700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4105cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.524+0000 7f49d9a19700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4105cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35262/0 (socket says 172.21.15.100:35262) 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.524+0000 7f49d9a19700 1 -- 172.21.15.100:0/3419867245 learned_addr learned my addr 172.21.15.100:0/3419867245 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.524+0000 7f49d9a19700 1 -- 172.21.15.100:0/3419867245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0009550 con 0x7f49d4106470 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.525+0000 7f49d9a19700 1 --2- 172.21.15.100:0/3419867245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4105cf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f49d00119a0 tx=0x7f49d0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:08.386 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.525+0000 7f49caffd700 1 -- 172.21.15.100:0/3419867245 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d0011e90 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.525+0000 7f49dbc7d700 1 -- 172.21.15.100:0/3419867245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d4104420 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.525+0000 7f49caffd700 1 -- 172.21.15.100:0/3419867245 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f49d00038b0 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.525+0000 7f49caffd700 1 -- 172.21.15.100:0/3419867245 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d0024e60 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.525+0000 7f49dbc7d700 1 -- 172.21.15.100:0/3419867245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d41048c0 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.526+0000 7f49caffd700 1 -- 172.21.15.100:0/3419867245 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f49d001a4c0 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.526+0000 7f49dbc7d700 1 -- 172.21.15.100:0/3419867245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d4109b20 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.526+0000 7f49caffd700 1 --2- 172.21.15.100:0/3419867245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f49c0038ee0 0x7f49c003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.526+0000 7f49caffd700 1 -- 172.21.15.100:0/3419867245 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f49d004ce90 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.526+0000 7f49d9218700 1 --2- 172.21.15.100:0/3419867245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f49c0038ee0 0x7f49c003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.526+0000 7f49d9218700 1 --2- 172.21.15.100:0/3419867245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f49c0038ee0 0x7f49c003b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49c4006fd0 tx=0x7f49c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.531+0000 7f49caffd700 1 -- 172.21.15.100:0/3419867245 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49d0018c20 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.731+0000 7f49dbc7d700 1 -- 172.21.15.100:0/3419867245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f49d4109d30 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.733+0000 7f49caffd700 1 -- 172.21.15.100:0/3419867245 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f49d0018350 con 0x7f49d4106470 2024-02-14T17:40:08.387 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.736+0000 7f49dbc7d700 1 -- 172.21.15.100:0/3419867245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f49c0038ee0 msgr2=0x7f49c003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:08.388 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.736+0000 7f49dbc7d700 1 --2- 172.21.15.100:0/3419867245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f49c0038ee0 0x7f49c003b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49c4006fd0 tx=0x7f49c4006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:08.388 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.736+0000 7f49dbc7d700 1 -- 172.21.15.100:0/3419867245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 msgr2=0x7f49d4105cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:08.388 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.736+0000 7f49dbc7d700 1 --2- 172.21.15.100:0/3419867245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4105cf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f49d00119a0 tx=0x7f49d0011b70 comp rx=0 tx=0).stop 2024-02-14T17:40:08.388 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.736+0000 7f49dbc7d700 1 -- 172.21.15.100:0/3419867245 shutdown_connections 2024-02-14T17:40:08.388 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.736+0000 7f49dbc7d700 1 --2- 172.21.15.100:0/3419867245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f49c0038ee0 0x7f49c003b3a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:08.388 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.736+0000 7f49dbc7d700 1 --2- 172.21.15.100:0/3419867245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49d4106470 0x7f49d4105cf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:08.388 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.736+0000 7f49dbc7d700 1 -- 172.21.15.100:0/3419867245 >> 172.21.15.100:0/3419867245 conn(0x7f49d40ffec0 msgr2=0x7f49d41009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:08.388 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.736+0000 7f49dbc7d700 1 -- 172.21.15.100:0/3419867245 shutdown_connections 2024-02-14T17:40:08.388 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:07.736+0000 7f49dbc7d700 1 -- 172.21.15.100:0/3419867245 wait complete. 2024-02-14T17:40:09.060 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3419867245' entity='client.admin' 2024-02-14T17:40:09.793 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.862+0000 7f7044800700 1 Processor -- start 2024-02-14T17:40:09.794 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.863+0000 7f7044800700 1 -- start start 2024-02-14T17:40:09.794 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.863+0000 7f7044800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f7040104b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:09.794 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.863+0000 7f7044800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7040105150 con 0x7f7040104760 2024-02-14T17:40:09.794 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.863+0000 7f703ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f7040104b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:09.794 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.863+0000 7f703ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f7040104b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35266/0 (socket says 172.21.15.100:35266) 2024-02-14T17:40:09.794 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.863+0000 7f703ed9d700 1 -- 172.21.15.100:0/3073167614 learned_addr learned my addr 172.21.15.100:0/3073167614 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:09.795 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.863+0000 7f703ed9d700 1 -- 172.21.15.100:0/3073167614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040105960 con 0x7f7040104760 2024-02-14T17:40:09.795 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.864+0000 7f703ed9d700 1 --2- 172.21.15.100:0/3073167614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f7040104b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f70300098a0 tx=0x7f7030009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceeb94fe5d5d00b server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:09.795 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.864+0000 7f703dd9b700 1 -- 172.21.15.100:0/3073167614 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7030004160 con 0x7f7040104760 2024-02-14T17:40:09.795 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.864+0000 7f703dd9b700 1 -- 172.21.15.100:0/3073167614 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f70300042c0 con 0x7f7040104760 2024-02-14T17:40:09.795 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.864+0000 7f703dd9b700 1 -- 172.21.15.100:0/3073167614 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7030011850 con 0x7f7040104760 2024-02-14T17:40:09.795 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.865+0000 7f7044800700 1 -- 172.21.15.100:0/3073167614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 msgr2=0x7f7040104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:09.796 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.865+0000 7f7044800700 1 --2- 172.21.15.100:0/3073167614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f7040104b80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f70300098a0 tx=0x7f7030009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:09.796 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.865+0000 7f7044800700 1 -- 172.21.15.100:0/3073167614 shutdown_connections 2024-02-14T17:40:09.796 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.865+0000 7f7044800700 1 --2- 172.21.15.100:0/3073167614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f7040104b80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:09.796 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.865+0000 7f7044800700 1 -- 172.21.15.100:0/3073167614 >> 172.21.15.100:0/3073167614 conn(0x7f70400ffcc0 msgr2=0x7f7040102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:09.796 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.865+0000 7f7044800700 1 -- 172.21.15.100:0/3073167614 shutdown_connections 2024-02-14T17:40:09.796 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.865+0000 7f7044800700 1 -- 172.21.15.100:0/3073167614 wait complete. 2024-02-14T17:40:09.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.866+0000 7f7044800700 1 Processor -- start 2024-02-14T17:40:09.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.866+0000 7f7044800700 1 -- start start 2024-02-14T17:40:09.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.866+0000 7f7044800700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f70401983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:09.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.866+0000 7f7044800700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70401988f0 con 0x7f7040104760 2024-02-14T17:40:09.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.866+0000 7f703ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f70401983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:09.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.866+0000 7f703ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f70401983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35278/0 (socket says 172.21.15.100:35278) 2024-02-14T17:40:09.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.866+0000 7f703ed9d700 1 -- 172.21.15.100:0/2246539245 learned_addr learned my addr 172.21.15.100:0/2246539245 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:09.797 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.867+0000 7f703ed9d700 1 -- 172.21.15.100:0/2246539245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7030009550 con 0x7f7040104760 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.867+0000 7f703ed9d700 1 --2- 172.21.15.100:0/2246539245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f70401983b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7030003d50 tx=0x7f7030011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.867+0000 7f7027fff700 1 -- 172.21.15.100:0/2246539245 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70300037d0 con 0x7f7040104760 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.867+0000 7f7027fff700 1 -- 172.21.15.100:0/2246539245 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f7030003930 con 0x7f7040104760 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.867+0000 7f7027fff700 1 -- 172.21.15.100:0/2246539245 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70300193c0 con 0x7f7040104760 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.868+0000 7f7044800700 1 -- 172.21.15.100:0/2246539245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7040198af0 con 0x7f7040104760 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.868+0000 7f7044800700 1 -- 172.21.15.100:0/2246539245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7040198f90 con 0x7f7040104760 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.869+0000 7f7027fff700 1 -- 172.21.15.100:0/2246539245 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f703001a430 con 0x7f7040104760 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.869+0000 7f7044800700 1 -- 172.21.15.100:0/2246539245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70400632a0 con 0x7f7040104760 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.869+0000 7f7027fff700 1 --2- 172.21.15.100:0/2246539245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f702803b0f0 0x7f702803d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.869+0000 7f7027fff700 1 -- 172.21.15.100:0/2246539245 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f703004cf90 con 0x7f7040104760 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.869+0000 7f703e59c700 1 --2- 172.21.15.100:0/2246539245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f702803b0f0 0x7f702803d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:09.798 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.870+0000 7f703e59c700 1 --2- 172.21.15.100:0/2246539245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f702803b0f0 0x7f702803d5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7034006fd0 tx=0x7f7034006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:08.875+0000 7f7027fff700 1 -- 172.21.15.100:0/2246539245 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7030080d20 con 0x7f7040104760 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.087+0000 7f7044800700 1 -- 172.21.15.100:0/2246539245 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f7040078990 con 0x7f7040104760 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.090+0000 7f7027fff700 1 -- 172.21.15.100:0/2246539245 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f7030003aa0 con 0x7f7040104760 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.092+0000 7f7044800700 1 -- 172.21.15.100:0/2246539245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f702803b0f0 msgr2=0x7f702803d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.092+0000 7f7044800700 1 --2- 172.21.15.100:0/2246539245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f702803b0f0 0x7f702803d5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7034006fd0 tx=0x7f7034006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.092+0000 7f7044800700 1 -- 172.21.15.100:0/2246539245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 msgr2=0x7f70401983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.092+0000 7f7044800700 1 --2- 172.21.15.100:0/2246539245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f70401983b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7030003d50 tx=0x7f7030011ac0 comp rx=0 tx=0).stop 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.093+0000 7f7044800700 1 -- 172.21.15.100:0/2246539245 shutdown_connections 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.093+0000 7f7044800700 1 --2- 172.21.15.100:0/2246539245 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f702803b0f0 0x7f702803d5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.093+0000 7f7044800700 1 --2- 172.21.15.100:0/2246539245 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7040104760 0x7f70401983b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.093+0000 7f7044800700 1 -- 172.21.15.100:0/2246539245 >> 172.21.15.100:0/2246539245 conn(0x7f70400ffcc0 msgr2=0x7f70401007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.093+0000 7f7044800700 1 -- 172.21.15.100:0/2246539245 shutdown_connections 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:09.093+0000 7f7044800700 1 -- 172.21.15.100:0/2246539245 wait complete. 2024-02-14T17:40:09.799 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling the dashboard module... 2024-02-14T17:40:10.383 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2246539245' entity='client.admin' 2024-02-14T17:40:11.350 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2323188516' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-14T17:40:11.351 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:11 smithi100 ceph-mon[30138]: from='mgr.14120 172.21.15.100:0/4126006264' entity='mgr.y' 2024-02-14T17:40:11.351 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:11 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ignoring --setuser ceph since I am not root 2024-02-14T17:40:11.351 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:11 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ignoring --setgroup ceph since I am not root 2024-02-14T17:40:11.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.381+0000 7f024c3dc700 1 Processor -- start 2024-02-14T17:40:11.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.381+0000 7f024c3dc700 1 -- start start 2024-02-14T17:40:11.784 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.381+0000 7f024c3dc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f0244079f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.381+0000 7f024c3dc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024407b8a0 con 0x7f0244079b70 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.382+0000 7f024a178700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f0244079f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.382+0000 7f024a178700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f0244079f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35288/0 (socket says 172.21.15.100:35288) 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.382+0000 7f024a178700 1 -- 172.21.15.100:0/3861714131 learned_addr learned my addr 172.21.15.100:0/3861714131 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.382+0000 7f024a178700 1 -- 172.21.15.100:0/3861714131 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024407a4d0 con 0x7f0244079b70 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.382+0000 7f024a178700 1 --2- 172.21.15.100:0/3861714131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f0244079f90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f0240009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2240cf3d145ed3 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.382+0000 7f0249176700 1 -- 172.21.15.100:0/3861714131 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0240004160 con 0x7f0244079b70 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.382+0000 7f0249176700 1 -- 172.21.15.100:0/3861714131 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f02400042c0 con 0x7f0244079b70 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.382+0000 7f0249176700 1 -- 172.21.15.100:0/3861714131 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0240011850 con 0x7f0244079b70 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.383+0000 7f024c3dc700 1 -- 172.21.15.100:0/3861714131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 msgr2=0x7f0244079f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.383+0000 7f024c3dc700 1 --2- 172.21.15.100:0/3861714131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f0244079f90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f0240009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.383+0000 7f024c3dc700 1 -- 172.21.15.100:0/3861714131 shutdown_connections 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.383+0000 7f024c3dc700 1 --2- 172.21.15.100:0/3861714131 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f0244079f90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.383+0000 7f024c3dc700 1 -- 172.21.15.100:0/3861714131 >> 172.21.15.100:0/3861714131 conn(0x7f0244101030 msgr2=0x7f0244103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.383+0000 7f024c3dc700 1 -- 172.21.15.100:0/3861714131 shutdown_connections 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.383+0000 7f024c3dc700 1 -- 172.21.15.100:0/3861714131 wait complete. 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.383+0000 7f024c3dc700 1 Processor -- start 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.383+0000 7f024c3dc700 1 -- start start 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f024c3dc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f024419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f024c3dc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024419be60 con 0x7f0244079b70 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f024a178700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f024419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f024a178700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f024419b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35304/0 (socket says 172.21.15.100:35304) 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f024a178700 1 -- 172.21.15.100:0/2323188516 learned_addr learned my addr 172.21.15.100:0/2323188516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f024a178700 1 -- 172.21.15.100:0/2323188516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0240009550 con 0x7f0244079b70 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f024a178700 1 --2- 172.21.15.100:0/2323188516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f024419b920 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0240003ec0 tx=0x7f0240011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:11.786 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f023b7fe700 1 -- 172.21.15.100:0/2323188516 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02400037d0 con 0x7f0244079b70 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f023b7fe700 1 -- 172.21.15.100:0/2323188516 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0240003930 con 0x7f0244079b70 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f023b7fe700 1 -- 172.21.15.100:0/2323188516 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02400193c0 con 0x7f0244079b70 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f024c3dc700 1 -- 172.21.15.100:0/2323188516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024419c060 con 0x7f0244079b70 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.384+0000 7f024c3dc700 1 -- 172.21.15.100:0/2323188516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024419c500 con 0x7f0244079b70 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.385+0000 7f023b7fe700 1 -- 172.21.15.100:0/2323188516 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f024001a430 con 0x7f0244079b70 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.385+0000 7f024c3dc700 1 -- 172.21.15.100:0/2323188516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024410be70 con 0x7f0244079b70 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.385+0000 7f023b7fe700 1 --2- 172.21.15.100:0/2323188516 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0230038f30 0x7f023003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.385+0000 7f023b7fe700 1 -- 172.21.15.100:0/2323188516 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f024004cf80 con 0x7f0244079b70 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.385+0000 7f0249977700 1 --2- 172.21.15.100:0/2323188516 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0230038f30 0x7f023003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.386+0000 7f0249977700 1 --2- 172.21.15.100:0/2323188516 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0230038f30 0x7f023003b3f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f023400ad30 tx=0x7f02340093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.390+0000 7f023b7fe700 1 -- 172.21.15.100:0/2323188516 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0240003aa0 con 0x7f0244079b70 2024-02-14T17:40:11.787 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:10.639+0000 7f024c3dc700 1 -- 172.21.15.100:0/2323188516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f024419ede0 con 0x7f0244079b70 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.093+0000 7f023b7fe700 1 -- 172.21.15.100:0/2323188516 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f024002d350 con 0x7f0244079b70 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.095+0000 7f023b7fe700 1 -- 172.21.15.100:0/2323188516 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f024002bd00 con 0x7f0244079b70 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.097+0000 7f024c3dc700 1 -- 172.21.15.100:0/2323188516 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0230038f30 msgr2=0x7f023003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.097+0000 7f024c3dc700 1 --2- 172.21.15.100:0/2323188516 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0230038f30 0x7f023003b3f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f023400ad30 tx=0x7f02340093f0 comp rx=0 tx=0).stop 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.097+0000 7f024c3dc700 1 -- 172.21.15.100:0/2323188516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 msgr2=0x7f024419b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.097+0000 7f024c3dc700 1 --2- 172.21.15.100:0/2323188516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f024419b920 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0240003ec0 tx=0x7f0240011980 comp rx=0 tx=0).stop 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.097+0000 7f024c3dc700 1 -- 172.21.15.100:0/2323188516 shutdown_connections 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.097+0000 7f024c3dc700 1 --2- 172.21.15.100:0/2323188516 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0230038f30 0x7f023003b3f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.097+0000 7f024c3dc700 1 --2- 172.21.15.100:0/2323188516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0244079b70 0x7f024419b920 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.097+0000 7f024c3dc700 1 -- 172.21.15.100:0/2323188516 >> 172.21.15.100:0/2323188516 conn(0x7f0244101030 msgr2=0x7f0244101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:11.788 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.097+0000 7f024c3dc700 1 -- 172.21.15.100:0/2323188516 shutdown_connections 2024-02-14T17:40:11.789 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:11.097+0000 7f024c3dc700 1 -- 172.21.15.100:0/2323188516 wait complete. 2024-02-14T17:40:12.037 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:11 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:11.677+0000 7f4a35eff200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:40:12.037 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:11 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:11.772+0000 7f4a35eff200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:40:12.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2323188516' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-14T17:40:12.295 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:12 smithi100 ceph-mon[30138]: mgrmap e9: y(active, since 20s) 2024-02-14T17:40:12.550 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:12 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:12.453+0000 7f4a35eff200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:40:12.550 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:12 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:12.548+0000 7f4a35eff200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:40:12.878 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:12 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:12.694+0000 7f4a35eff200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:40:12.878 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:12 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:12.790+0000 7f4a35eff200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:40:12.878 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:12 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:12.876+0000 7f4a35eff200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:40:13.143 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2325295678' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-14T17:40:13.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:40:13.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-14T17:40:13.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-14T17:40:13.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-14T17:40:13.191 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-14T17:40:13.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:40:13.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.310+0000 7f0f9d26e700 1 Processor -- start 2024-02-14T17:40:13.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.310+0000 7f0f9d26e700 1 -- start start 2024-02-14T17:40:13.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.310+0000 7f0f9d26e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f981088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:13.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.310+0000 7f0f9d26e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f98078e50 con 0x7f0f981064b0 2024-02-14T17:40:13.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.311+0000 7f0f96ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f981088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:13.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.311+0000 7f0f96ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f981088a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35322/0 (socket says 172.21.15.100:35322) 2024-02-14T17:40:13.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.311+0000 7f0f96ffd700 1 -- 172.21.15.100:0/796854085 learned_addr learned my addr 172.21.15.100:0/796854085 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:13.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.311+0000 7f0f96ffd700 1 -- 172.21.15.100:0/796854085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f98108de0 con 0x7f0f981064b0 2024-02-14T17:40:13.192 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.311+0000 7f0f96ffd700 1 --2- 172.21.15.100:0/796854085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f981088a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f880098a0 tx=0x7f0f88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8ec04ad9acb2d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.311+0000 7f0f95ffb700 1 -- 172.21.15.100:0/796854085 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f88004160 con 0x7f0f981064b0 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.311+0000 7f0f95ffb700 1 -- 172.21.15.100:0/796854085 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0f880042c0 con 0x7f0f981064b0 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.311+0000 7f0f95ffb700 1 -- 172.21.15.100:0/796854085 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f88011850 con 0x7f0f981064b0 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.312+0000 7f0f9d26e700 1 -- 172.21.15.100:0/796854085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 msgr2=0x7f0f981088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.312+0000 7f0f9d26e700 1 --2- 172.21.15.100:0/796854085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f981088a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f880098a0 tx=0x7f0f88009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.312+0000 7f0f9d26e700 1 -- 172.21.15.100:0/796854085 shutdown_connections 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.312+0000 7f0f9d26e700 1 --2- 172.21.15.100:0/796854085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f981088a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.312+0000 7f0f9d26e700 1 -- 172.21.15.100:0/796854085 >> 172.21.15.100:0/796854085 conn(0x7f0f980ffec0 msgr2=0x7f0f981022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.312+0000 7f0f9d26e700 1 -- 172.21.15.100:0/796854085 shutdown_connections 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.312+0000 7f0f9d26e700 1 -- 172.21.15.100:0/796854085 wait complete. 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.313+0000 7f0f9d26e700 1 Processor -- start 2024-02-14T17:40:13.193 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.313+0000 7f0f9d26e700 1 -- start start 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.313+0000 7f0f9d26e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f98105d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.313+0000 7f0f9d26e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f98106270 con 0x7f0f981064b0 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.313+0000 7f0f96ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f98105d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.313+0000 7f0f96ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f98105d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35326/0 (socket says 172.21.15.100:35326) 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.313+0000 7f0f96ffd700 1 -- 172.21.15.100:0/2325295678 learned_addr learned my addr 172.21.15.100:0/2325295678 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.313+0000 7f0f96ffd700 1 -- 172.21.15.100:0/2325295678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88009550 con 0x7f0f981064b0 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.314+0000 7f0f96ffd700 1 --2- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f98105d30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0f880119a0 tx=0x7f0f88011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.314+0000 7f0f8ffff700 1 -- 172.21.15.100:0/2325295678 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f88011e90 con 0x7f0f981064b0 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.314+0000 7f0f8ffff700 1 -- 172.21.15.100:0/2325295678 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0f880038b0 con 0x7f0f981064b0 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.314+0000 7f0f8ffff700 1 -- 172.21.15.100:0/2325295678 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f88024e60 con 0x7f0f981064b0 2024-02-14T17:40:13.194 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.314+0000 7f0f9d26e700 1 -- 172.21.15.100:0/2325295678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f98104460 con 0x7f0f981064b0 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.314+0000 7f0f9d26e700 1 -- 172.21.15.100:0/2325295678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f98104900 con 0x7f0f981064b0 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.315+0000 7f0f8ffff700 1 -- 172.21.15.100:0/2325295678 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f0f8801a480 con 0x7f0f981064b0 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.315+0000 7f0f8ffff700 1 --2- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0f84038f90 0x7f0f8403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.315+0000 7f0f8ffff700 1 -- 172.21.15.100:0/2325295678 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0f8804d000 con 0x7f0f981064b0 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.315+0000 7f0f967fc700 1 -- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0f84038f90 msgr2=0x7f0f8403b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1559623009 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.315+0000 7f0f967fc700 1 --2- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0f84038f90 0x7f0f8403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.315+0000 7f0f9d26e700 1 -- 172.21.15.100:0/2325295678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f980632a0 con 0x7f0f981064b0 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.320+0000 7f0f8ffff700 1 -- 172.21.15.100:0/2325295678 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f8802d3f0 con 0x7f0f981064b0 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.516+0000 7f0f967fc700 1 -- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0f84038f90 msgr2=0x7f0f8403b450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1559623009 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.516+0000 7f0f967fc700 1 --2- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0f84038f90 0x7f0f8403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T17:40:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.582+0000 7f0f9d26e700 1 -- 172.21.15.100:0/2325295678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0f98109cb0 con 0x7f0f981064b0 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.583+0000 7f0f8ffff700 1 -- 172.21.15.100:0/2325295678 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f0f88018350 con 0x7f0f981064b0 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.585+0000 7f0f9d26e700 1 -- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0f84038f90 msgr2=0x7f0f8403b450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.585+0000 7f0f9d26e700 1 --2- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0f84038f90 0x7f0f8403b450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.585+0000 7f0f9d26e700 1 -- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 msgr2=0x7f0f98105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.585+0000 7f0f9d26e700 1 --2- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f98105d30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0f880119a0 tx=0x7f0f88011b70 comp rx=0 tx=0).stop 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.585+0000 7f0f9d26e700 1 -- 172.21.15.100:0/2325295678 shutdown_connections 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.585+0000 7f0f9d26e700 1 --2- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7f0f84038f90 0x7f0f8403b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.585+0000 7f0f9d26e700 1 --2- 172.21.15.100:0/2325295678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0f981064b0 0x7f0f98105d30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.585+0000 7f0f9d26e700 1 -- 172.21.15.100:0/2325295678 >> 172.21.15.100:0/2325295678 conn(0x7f0f980ffec0 msgr2=0x7f0f98100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.585+0000 7f0f9d26e700 1 -- 172.21.15.100:0/2325295678 shutdown_connections 2024-02-14T17:40:13.196 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:12.585+0000 7f0f9d26e700 1 -- 172.21.15.100:0/2325295678 wait complete. 2024-02-14T17:40:13.197 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for the mgr to restart... 2024-02-14T17:40:13.197 INFO:teuthology.orchestra.run.smithi100.stdout:Waiting for mgr epoch 9... 2024-02-14T17:40:13.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:13 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:13.141+0000 7f4a35eff200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:40:13.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:13 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:13.256+0000 7f4a35eff200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:40:13.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:13 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:13.342+0000 7f4a35eff200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:40:13.705 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:13 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:13.440+0000 7f4a35eff200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:40:13.705 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:13 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:13.623+0000 7f4a35eff200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:40:14.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:14 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:14.597+0000 7f4a35eff200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:40:14.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:14 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:14.729+0000 7f4a35eff200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:40:15.358 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:15 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:15.068+0000 7f4a35eff200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:40:15.359 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:15 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:15.159+0000 7f4a35eff200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:40:15.359 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:15 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:15.356+0000 7f4a35eff200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:40:15.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:15 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:15.543+0000 7f4a35eff200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:40:16.086 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:15 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:15.827+0000 7f4a35eff200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:40:16.087 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:16 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:16.084+0000 7f4a35eff200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:40:16.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:16 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:16.319+0000 7f4a35eff200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:40:17.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:17 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:17.118+0000 7f4a35eff200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:40:18.001 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:17 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:17.685+0000 7f4a35eff200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:40:18.002 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:17 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:17.783+0000 7f4a35eff200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:40:18.002 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:40:17.999+0000 7f4a35eff200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:40:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: Active manager daemon y restarted 2024-02-14T17:40:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: Activating manager daemon y 2024-02-14T17:40:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: osdmap e3: 0 total, 0 up, 0 in 2024-02-14T17:40:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: mgrmap e10: y(active, starting, since 0.00329455s) 2024-02-14T17:40:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:40:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:40:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:40:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:40:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:40:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: Manager daemon y is now available 2024-02-14T17:40:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:19.331 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:19 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:40:19.331 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:19 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:40:19.331 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:19 smithi100 ceph-mon[30138]: mgrmap e11: y(active, since 1.00753s) 2024-02-14T17:40:19.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout { 2024-02-14T17:40:19.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-14T17:40:19.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-14T17:40:19.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout } 2024-02-14T17:40:19.625 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.703+0000 7fa2151e2700 1 Processor -- start 2024-02-14T17:40:19.626 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.703+0000 7fa2151e2700 1 -- start start 2024-02-14T17:40:19.626 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.703+0000 7fa2151e2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:19.626 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.703+0000 7fa2151e2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa210079720 con 0x7fa2101064a0 2024-02-14T17:40:19.626 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.703+0000 7fa20ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:19.626 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.703+0000 7fa20ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36744/0 (socket says 172.21.15.100:36744) 2024-02-14T17:40:19.626 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.703+0000 7fa20ed9d700 1 -- 172.21.15.100:0/2395045298 learned_addr learned my addr 172.21.15.100:0/2395045298 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:19.626 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.703+0000 7fa20ed9d700 1 -- 172.21.15.100:0/2395045298 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa210108dd0 con 0x7fa2101064a0 2024-02-14T17:40:19.626 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa20ed9d700 1 --2- 172.21.15.100:0/2395045298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210108890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2000098a0 tx=0x7fa200009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d598c5cceb7903c server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:19.626 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa20dd9b700 1 -- 172.21.15.100:0/2395045298 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa200004160 con 0x7fa2101064a0 2024-02-14T17:40:19.626 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa20dd9b700 1 -- 172.21.15.100:0/2395045298 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa2000042c0 con 0x7fa2101064a0 2024-02-14T17:40:19.627 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa20dd9b700 1 -- 172.21.15.100:0/2395045298 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa200011850 con 0x7fa2101064a0 2024-02-14T17:40:19.627 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa2151e2700 1 -- 172.21.15.100:0/2395045298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 msgr2=0x7fa210108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:19.627 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa2151e2700 1 --2- 172.21.15.100:0/2395045298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210108890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2000098a0 tx=0x7fa200009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:19.627 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa2151e2700 1 -- 172.21.15.100:0/2395045298 shutdown_connections 2024-02-14T17:40:19.627 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa2151e2700 1 --2- 172.21.15.100:0/2395045298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210108890 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:19.627 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa2151e2700 1 -- 172.21.15.100:0/2395045298 >> 172.21.15.100:0/2395045298 conn(0x7fa2100ffef0 msgr2=0x7fa210102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa2151e2700 1 -- 172.21.15.100:0/2395045298 shutdown_connections 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.704+0000 7fa2151e2700 1 -- 172.21.15.100:0/2395045298 wait complete. 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.705+0000 7fa2151e2700 1 Processor -- start 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.705+0000 7fa2151e2700 1 -- start start 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.705+0000 7fa2151e2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.705+0000 7fa2151e2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa210199b90 con 0x7fa2101064a0 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.706+0000 7fa20ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.706+0000 7fa20ed9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36748/0 (socket says 172.21.15.100:36748) 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.706+0000 7fa20ed9d700 1 -- 172.21.15.100:0/1145376375 learned_addr learned my addr 172.21.15.100:0/1145376375 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.706+0000 7fa20ed9d700 1 -- 172.21.15.100:0/1145376375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa200009550 con 0x7fa2101064a0 2024-02-14T17:40:19.628 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.706+0000 7fa20ed9d700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210199650 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa200006af0 tx=0x7fa200011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.706+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2000036a0 con 0x7fa2101064a0 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.706+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa20001a430 con 0x7fa2101064a0 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.706+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2000193c0 con 0x7fa2101064a0 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.706+0000 7fa2151e2700 1 -- 172.21.15.100:0/1145376375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa210199d90 con 0x7fa2101064a0 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.706+0000 7fa2151e2700 1 -- 172.21.15.100:0/1145376375 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21019a230 con 0x7fa2101064a0 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.707+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fa200019680 con 0x7fa2101064a0 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.707+0000 7fa1f7fff700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 0x7fa1f803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.707+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 --> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1f803bab0 con 0x7fa1f8038ee0 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.707+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa20004cd10 con 0x7fa2101064a0 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.707+0000 7fa20e59c700 1 -- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 msgr2=0x7fa1f803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1559623009 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.707+0000 7fa20e59c700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 0x7fa1f803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.908+0000 7fa20e59c700 1 -- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 msgr2=0x7fa1f803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1559623009 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:13.908+0000 7fa20e59c700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 0x7fa1f803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T17:40:19.629 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:14.309+0000 7fa20e59c700 1 -- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 msgr2=0x7fa1f803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1559623009 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:14.309+0000 7fa20e59c700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 0x7fa1f803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:15.110+0000 7fa20e59c700 1 -- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 msgr2=0x7fa1f803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1559623009 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:15.110+0000 7fa20e59c700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 0x7fa1f803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:16.711+0000 7fa20e59c700 1 -- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 msgr2=0x7fa1f803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/1559623009 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:16.711+0000 7fa20e59c700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 0x7fa1f803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:18.015+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7fa200019c00 con 0x7fa2101064a0 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:18.015+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 msgr2=0x7fa1f803b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:18.015+0000 7fa1f7fff700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 0x7fa1f803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.017+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7fa20000fda0 con 0x7fa2101064a0 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.017+0000 7fa1f7fff700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1f803c960 0x7fa1f803ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.017+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1f803bab0 con 0x7fa1f803c960 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.018+0000 7fa20e59c700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1f803c960 0x7fa1f803ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.018+0000 7fa20e59c700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1f803c960 0x7fa1f803ed50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa204003a10 tx=0x7fa2040092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:19.630 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.019+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fa1f803bab0 con 0x7fa1f803c960 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.025+0000 7fa2151e2700 1 -- 172.21.15.100:0/1145376375 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa210109b50 con 0x7fa1f803c960 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa1f7fff700 1 -- 172.21.15.100:0/1145376375 <== mgr.14152 v2:172.21.15.100:6800/3957561430 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa210109b50 con 0x7fa1f803c960 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 -- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1f803c960 msgr2=0x7fa1f803ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1f803c960 0x7fa1f803ed50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa204003a10 tx=0x7fa2040092b0 comp rx=0 tx=0).stop 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 -- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 msgr2=0x7fa210199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210199650 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa200006af0 tx=0x7fa200011a50 comp rx=0 tx=0).stop 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 -- 172.21.15.100:0/1145376375 shutdown_connections 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1f803c960 0x7fa1f803ed50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2101064a0 0x7fa210199650 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 --2- 172.21.15.100:0/1145376375 >> [v2:172.21.15.100:6800/1559623009,v1:172.21.15.100:6801/1559623009] conn(0x7fa1f8038ee0 0x7fa1f803b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 -- 172.21.15.100:0/1145376375 >> 172.21.15.100:0/1145376375 conn(0x7fa2100ffef0 msgr2=0x7fa210100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 -- 172.21.15.100:0/1145376375 shutdown_connections 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:19.026+0000 7fa2151e2700 1 -- 172.21.15.100:0/1145376375 wait complete. 2024-02-14T17:40:19.631 INFO:teuthology.orchestra.run.smithi100.stdout:mgr epoch 9 is available 2024-02-14T17:40:19.632 INFO:teuthology.orchestra.run.smithi100.stdout:Generating a dashboard self-signed certificate... 2024-02-14T17:40:20.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:20 smithi100 ceph-mon[30138]: [14/Feb/2024:17:40:18] ENGINE Bus STARTING 2024-02-14T17:40:20.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:20 smithi100 ceph-mon[30138]: [14/Feb/2024:17:40:19] ENGINE Serving on http://172.21.15.100:8765 2024-02-14T17:40:20.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:20 smithi100 ceph-mon[30138]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-14T17:40:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:20 smithi100 ceph-mon[30138]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-14T17:40:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:20 smithi100 ceph-mon[30138]: [14/Feb/2024:17:40:19] ENGINE Serving on https://172.21.15.100:7150 2024-02-14T17:40:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:20 smithi100 ceph-mon[30138]: [14/Feb/2024:17:40:19] ENGINE Bus STARTED 2024-02-14T17:40:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:21 smithi100 ceph-mon[30138]: mgrmap e12: y(active, since 2s) 2024-02-14T17:40:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:21.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-14T17:40:21.509 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.456+0000 7f2370e85700 1 Processor -- start 2024-02-14T17:40:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.456+0000 7f2370e85700 1 -- start start 2024-02-14T17:40:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.456+0000 7f2370e85700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c104100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.456+0000 7f2370e85700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c1046d0 con 0x7f236c103ce0 2024-02-14T17:40:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.456+0000 7f236a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.456+0000 7f236a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c104100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36806/0 (socket says 172.21.15.100:36806) 2024-02-14T17:40:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.456+0000 7f236a59c700 1 -- 172.21.15.100:0/119748559 learned_addr learned my addr 172.21.15.100:0/119748559 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.457+0000 7f236a59c700 1 -- 172.21.15.100:0/119748559 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c104810 con 0x7f236c103ce0 2024-02-14T17:40:21.510 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.457+0000 7f236a59c700 1 --2- 172.21.15.100:0/119748559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c104100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f235c0098a0 tx=0x7f235c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb918b5db2a0a8a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:21.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.457+0000 7f236959a700 1 -- 172.21.15.100:0/119748559 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f235c004160 con 0x7f236c103ce0 2024-02-14T17:40:21.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.457+0000 7f236959a700 1 -- 172.21.15.100:0/119748559 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f235c0042c0 con 0x7f236c103ce0 2024-02-14T17:40:21.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.457+0000 7f236959a700 1 -- 172.21.15.100:0/119748559 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f235c011850 con 0x7f236c103ce0 2024-02-14T17:40:21.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.458+0000 7f2370e85700 1 -- 172.21.15.100:0/119748559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 msgr2=0x7f236c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:21.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.458+0000 7f2370e85700 1 --2- 172.21.15.100:0/119748559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c104100 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f235c0098a0 tx=0x7f235c009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:21.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.458+0000 7f2370e85700 1 -- 172.21.15.100:0/119748559 shutdown_connections 2024-02-14T17:40:21.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.458+0000 7f2370e85700 1 --2- 172.21.15.100:0/119748559 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c104100 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:21.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.458+0000 7f2370e85700 1 -- 172.21.15.100:0/119748559 >> 172.21.15.100:0/119748559 conn(0x7f236c0ff640 msgr2=0x7f236c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:21.511 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.458+0000 7f2370e85700 1 -- 172.21.15.100:0/119748559 shutdown_connections 2024-02-14T17:40:21.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.458+0000 7f2370e85700 1 -- 172.21.15.100:0/119748559 wait complete. 2024-02-14T17:40:21.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.459+0000 7f2370e85700 1 Processor -- start 2024-02-14T17:40:21.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.459+0000 7f2370e85700 1 -- start start 2024-02-14T17:40:21.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.459+0000 7f2370e85700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:21.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.459+0000 7f2370e85700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c199be0 con 0x7f236c103ce0 2024-02-14T17:40:21.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.459+0000 7f236a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:21.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.459+0000 7f236a59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36818/0 (socket says 172.21.15.100:36818) 2024-02-14T17:40:21.512 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.459+0000 7f236a59c700 1 -- 172.21.15.100:0/1363320932 learned_addr learned my addr 172.21.15.100:0/1363320932 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:21.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.459+0000 7f236a59c700 1 -- 172.21.15.100:0/1363320932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c009550 con 0x7f236c103ce0 2024-02-14T17:40:21.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.459+0000 7f236a59c700 1 --2- 172.21.15.100:0/1363320932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c1996a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f235c000c00 tx=0x7f235c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:21.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.459+0000 7f235b7fe700 1 -- 172.21.15.100:0/1363320932 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f235c0036a0 con 0x7f236c103ce0 2024-02-14T17:40:21.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.460+0000 7f235b7fe700 1 -- 172.21.15.100:0/1363320932 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f235c024440 con 0x7f236c103ce0 2024-02-14T17:40:21.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.460+0000 7f2370e85700 1 -- 172.21.15.100:0/1363320932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c199e40 con 0x7f236c103ce0 2024-02-14T17:40:21.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.460+0000 7f235b7fe700 1 -- 172.21.15.100:0/1363320932 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f235c01a430 con 0x7f236c103ce0 2024-02-14T17:40:21.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.460+0000 7f2370e85700 1 -- 172.21.15.100:0/1363320932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c19cb60 con 0x7f236c103ce0 2024-02-14T17:40:21.513 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.461+0000 7f235b7fe700 1 -- 172.21.15.100:0/1363320932 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f235c01a900 con 0x7f236c103ce0 2024-02-14T17:40:21.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.461+0000 7f2370e85700 1 -- 172.21.15.100:0/1363320932 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c109f90 con 0x7f236c103ce0 2024-02-14T17:40:21.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.461+0000 7f235b7fe700 1 --2- 172.21.15.100:0/1363320932 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f2354038f30 0x7f235403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:21.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.461+0000 7f235b7fe700 1 -- 172.21.15.100:0/1363320932 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f235c04d130 con 0x7f236c103ce0 2024-02-14T17:40:21.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.461+0000 7f2369d9b700 1 --2- 172.21.15.100:0/1363320932 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f2354038f30 0x7f235403b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:21.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.461+0000 7f2369d9b700 1 --2- 172.21.15.100:0/1363320932 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f2354038f30 0x7f235403b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2360006fd0 tx=0x7f2360006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:21.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.466+0000 7f235b7fe700 1 -- 172.21.15.100:0/1363320932 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f235c0233a0 con 0x7f236c103ce0 2024-02-14T17:40:21.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.673+0000 7f2370e85700 1 -- 172.21.15.100:0/1363320932 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f236c100920 con 0x7f2354038f30 2024-02-14T17:40:21.514 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.762+0000 7f235b7fe700 1 -- 172.21.15.100:0/1363320932 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f236c100920 con 0x7f2354038f30 2024-02-14T17:40:21.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.765+0000 7f2370e85700 1 -- 172.21.15.100:0/1363320932 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f2354038f30 msgr2=0x7f235403b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:21.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.765+0000 7f2370e85700 1 --2- 172.21.15.100:0/1363320932 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f2354038f30 0x7f235403b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2360006fd0 tx=0x7f2360006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:21.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.765+0000 7f2370e85700 1 -- 172.21.15.100:0/1363320932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 msgr2=0x7f236c1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:21.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.765+0000 7f2370e85700 1 --2- 172.21.15.100:0/1363320932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c1996a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f235c000c00 tx=0x7f235c011ac0 comp rx=0 tx=0).stop 2024-02-14T17:40:21.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.765+0000 7f2370e85700 1 -- 172.21.15.100:0/1363320932 shutdown_connections 2024-02-14T17:40:21.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.765+0000 7f2370e85700 1 --2- 172.21.15.100:0/1363320932 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f2354038f30 0x7f235403b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:21.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.765+0000 7f2370e85700 1 --2- 172.21.15.100:0/1363320932 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f236c103ce0 0x7f236c1996a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:21.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.765+0000 7f2370e85700 1 -- 172.21.15.100:0/1363320932 >> 172.21.15.100:0/1363320932 conn(0x7f236c0ff640 msgr2=0x7f236c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:21.515 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.765+0000 7f2370e85700 1 -- 172.21.15.100:0/1363320932 shutdown_connections 2024-02-14T17:40:21.516 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:20.765+0000 7f2370e85700 1 -- 172.21.15.100:0/1363320932 wait complete. 2024-02-14T17:40:21.516 INFO:teuthology.orchestra.run.smithi100.stdout:Creating initial admin user... 2024-02-14T17:40:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:22 smithi100 ceph-mon[30138]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:22 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$WQFPhSDYUpB.XZpc9gnetujH51.xDqQ6MdiT/hvEXBq0x7uq/nkLy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707932422, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.998+0000 7fcc1c5f7700 1 Processor -- start 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.998+0000 7fcc1c5f7700 1 -- start start 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.998+0000 7fcc1c5f7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc141062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.998+0000 7fcc1c5f7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14106880 con 0x7fcc14105eb0 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.998+0000 7fcc1a393700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc141062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.998+0000 7fcc1a393700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc141062b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36832/0 (socket says 172.21.15.100:36832) 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.998+0000 7fcc1a393700 1 -- 172.21.15.100:0/3416655115 learned_addr learned my addr 172.21.15.100:0/3416655115 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.998+0000 7fcc1a393700 1 -- 172.21.15.100:0/3416655115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc141069c0 con 0x7fcc14105eb0 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.999+0000 7fcc1a393700 1 --2- 172.21.15.100:0/3416655115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc141062b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098a0 tx=0x7fcc10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba36832138ded862 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:23.343 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.999+0000 7fcc19391700 1 -- 172.21.15.100:0/3416655115 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc10004160 con 0x7fcc14105eb0 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.999+0000 7fcc19391700 1 -- 172.21.15.100:0/3416655115 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcc100042c0 con 0x7fcc14105eb0 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.999+0000 7fcc19391700 1 -- 172.21.15.100:0/3416655115 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc10011850 con 0x7fcc14105eb0 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.999+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/3416655115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 msgr2=0x7fcc141062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:21.999+0000 7fcc1c5f7700 1 --2- 172.21.15.100:0/3416655115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc141062b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098a0 tx=0x7fcc10009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/3416655115 shutdown_connections 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1c5f7700 1 --2- 172.21.15.100:0/3416655115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc141062b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/3416655115 >> 172.21.15.100:0/3416655115 conn(0x7fcc140ff640 msgr2=0x7fcc14101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/3416655115 shutdown_connections 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/3416655115 wait complete. 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1c5f7700 1 Processor -- start 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1c5f7700 1 -- start start 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1c5f7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc1419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:23.344 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1c5f7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1419be00 con 0x7fcc14105eb0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1a393700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc1419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1a393700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc1419b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36842/0 (socket says 172.21.15.100:36842) 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.000+0000 7fcc1a393700 1 -- 172.21.15.100:0/902821013 learned_addr learned my addr 172.21.15.100:0/902821013 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.001+0000 7fcc1a393700 1 -- 172.21.15.100:0/902821013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10009550 con 0x7fcc14105eb0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.001+0000 7fcc1a393700 1 --2- 172.21.15.100:0/902821013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc1419b8c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcc10006af0 tx=0x7fcc10011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.001+0000 7fcc0b7fe700 1 -- 172.21.15.100:0/902821013 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc100036a0 con 0x7fcc14105eb0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.001+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/902821013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1419c000 con 0x7fcc14105eb0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.001+0000 7fcc0b7fe700 1 -- 172.21.15.100:0/902821013 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcc1001a430 con 0x7fcc14105eb0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.001+0000 7fcc0b7fe700 1 -- 172.21.15.100:0/902821013 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc1001ae60 con 0x7fcc14105eb0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.001+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/902821013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1419c4a0 con 0x7fcc14105eb0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.002+0000 7fcc0b7fe700 1 -- 172.21.15.100:0/902821013 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fcc100194a0 con 0x7fcc14105eb0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.002+0000 7fcc0b7fe700 1 --2- 172.21.15.100:0/902821013 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcc00038ee0 0x7fcc0003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.002+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/902821013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc141045d0 con 0x7fcc14105eb0 2024-02-14T17:40:23.345 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.002+0000 7fcc0b7fe700 1 -- 172.21.15.100:0/902821013 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcc1004d010 con 0x7fcc14105eb0 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.002+0000 7fcc19b92700 1 --2- 172.21.15.100:0/902821013 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcc00038ee0 0x7fcc0003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.003+0000 7fcc19b92700 1 --2- 172.21.15.100:0/902821013 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcc00038ee0 0x7fcc0003b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc04006fd0 tx=0x7fcc04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.011+0000 7fcc0b7fe700 1 -- 172.21.15.100:0/902821013 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc1002d440 con 0x7fcc14105eb0 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.247+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/902821013 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fcc140630c0 con 0x7fcc00038ee0 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.475+0000 7fcc0b7fe700 1 -- 172.21.15.100:0/902821013 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fcc140630c0 con 0x7fcc00038ee0 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.477+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/902821013 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcc00038ee0 msgr2=0x7fcc0003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.477+0000 7fcc1c5f7700 1 --2- 172.21.15.100:0/902821013 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcc00038ee0 0x7fcc0003b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcc04006fd0 tx=0x7fcc04006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.477+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/902821013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 msgr2=0x7fcc1419b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.477+0000 7fcc1c5f7700 1 --2- 172.21.15.100:0/902821013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc1419b8c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcc10006af0 tx=0x7fcc10011a30 comp rx=0 tx=0).stop 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.477+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/902821013 shutdown_connections 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.477+0000 7fcc1c5f7700 1 --2- 172.21.15.100:0/902821013 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcc00038ee0 0x7fcc0003b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.477+0000 7fcc1c5f7700 1 --2- 172.21.15.100:0/902821013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc14105eb0 0x7fcc1419b8c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.477+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/902821013 >> 172.21.15.100:0/902821013 conn(0x7fcc140ff640 msgr2=0x7fcc14192050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:23.346 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.478+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/902821013 shutdown_connections 2024-02-14T17:40:23.347 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:22.478+0000 7fcc1c5f7700 1 -- 172.21.15.100:0/902821013 wait complete. 2024-02-14T17:40:23.347 INFO:teuthology.orchestra.run.smithi100.stdout:Fetching dashboard port number... 2024-02-14T17:40:23.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:23 smithi100 ceph-mon[30138]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:23.622 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2016016186' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-14T17:40:24.949 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stdout 8443 2024-02-14T17:40:24.949 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.846+0000 7fd2fafc3700 1 Processor -- start 2024-02-14T17:40:24.949 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.846+0000 7fd2fafc3700 1 -- start start 2024-02-14T17:40:24.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.846+0000 7fd2fafc3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f4108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:24.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.846+0000 7fd2fafc3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40797a0 con 0x7fd2f4106540 2024-02-14T17:40:24.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.846+0000 7fd2f8d5f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f4108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:24.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.846+0000 7fd2f8d5f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f4108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51408/0 (socket says 172.21.15.100:51408) 2024-02-14T17:40:24.950 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.846+0000 7fd2f8d5f700 1 -- 172.21.15.100:0/605349403 learned_addr learned my addr 172.21.15.100:0/605349403 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:24.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.846+0000 7fd2f8d5f700 1 -- 172.21.15.100:0/605349403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4108e70 con 0x7fd2f4106540 2024-02-14T17:40:24.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.846+0000 7fd2f8d5f700 1 --2- 172.21.15.100:0/605349403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f4108930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80098a0 tx=0x7fd2e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d720a45ce38e870a server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:24.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.847+0000 7fd2f37fe700 1 -- 172.21.15.100:0/605349403 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e8004160 con 0x7fd2f4106540 2024-02-14T17:40:24.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.847+0000 7fd2f37fe700 1 -- 172.21.15.100:0/605349403 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd2e80042c0 con 0x7fd2f4106540 2024-02-14T17:40:24.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.847+0000 7fd2f37fe700 1 -- 172.21.15.100:0/605349403 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e8011850 con 0x7fd2f4106540 2024-02-14T17:40:24.951 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.847+0000 7fd2fafc3700 1 -- 172.21.15.100:0/605349403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 msgr2=0x7fd2f4108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:24.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.847+0000 7fd2fafc3700 1 --2- 172.21.15.100:0/605349403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f4108930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80098a0 tx=0x7fd2e8009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:24.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.847+0000 7fd2fafc3700 1 -- 172.21.15.100:0/605349403 shutdown_connections 2024-02-14T17:40:24.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.847+0000 7fd2fafc3700 1 --2- 172.21.15.100:0/605349403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f4108930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:24.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.847+0000 7fd2fafc3700 1 -- 172.21.15.100:0/605349403 >> 172.21.15.100:0/605349403 conn(0x7fd2f40fff50 msgr2=0x7fd2f4102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:24.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.847+0000 7fd2fafc3700 1 -- 172.21.15.100:0/605349403 shutdown_connections 2024-02-14T17:40:24.952 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.847+0000 7fd2fafc3700 1 -- 172.21.15.100:0/605349403 wait complete. 2024-02-14T17:40:24.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.848+0000 7fd2fafc3700 1 Processor -- start 2024-02-14T17:40:24.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.848+0000 7fd2fafc3700 1 -- start start 2024-02-14T17:40:24.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.848+0000 7fd2fafc3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f41996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:24.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.848+0000 7fd2fafc3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4199c20 con 0x7fd2f4106540 2024-02-14T17:40:24.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.848+0000 7fd2f8d5f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f41996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:24.953 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.848+0000 7fd2f8d5f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f41996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51416/0 (socket says 172.21.15.100:51416) 2024-02-14T17:40:24.954 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.848+0000 7fd2f8d5f700 1 -- 172.21.15.100:0/2016016186 learned_addr learned my addr 172.21.15.100:0/2016016186 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:24.954 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.848+0000 7fd2f8d5f700 1 -- 172.21.15.100:0/2016016186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e8009550 con 0x7fd2f4106540 2024-02-14T17:40:24.954 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.848+0000 7fd2f8d5f700 1 --2- 172.21.15.100:0/2016016186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f41996e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8003ec0 tx=0x7fd2e8011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:24.954 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.849+0000 7fd2f1ffb700 1 -- 172.21.15.100:0/2016016186 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e80037d0 con 0x7fd2f4106540 2024-02-14T17:40:24.954 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.849+0000 7fd2f1ffb700 1 -- 172.21.15.100:0/2016016186 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd2e8003930 con 0x7fd2f4106540 2024-02-14T17:40:24.954 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.849+0000 7fd2fafc3700 1 -- 172.21.15.100:0/2016016186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f4199e80 con 0x7fd2f4106540 2024-02-14T17:40:24.954 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.849+0000 7fd2f1ffb700 1 -- 172.21.15.100:0/2016016186 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e80193c0 con 0x7fd2f4106540 2024-02-14T17:40:24.954 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.849+0000 7fd2fafc3700 1 -- 172.21.15.100:0/2016016186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f419cba0 con 0x7fd2f4106540 2024-02-14T17:40:24.954 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.850+0000 7fd2f1ffb700 1 -- 172.21.15.100:0/2016016186 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fd2e8019630 con 0x7fd2f4106540 2024-02-14T17:40:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.850+0000 7fd2fafc3700 1 -- 172.21.15.100:0/2016016186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f40632a0 con 0x7fd2f4106540 2024-02-14T17:40:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.850+0000 7fd2f1ffb700 1 --2- 172.21.15.100:0/2016016186 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd2e4038f30 0x7fd2e403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.850+0000 7fd2f1ffb700 1 -- 172.21.15.100:0/2016016186 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd2e804d0c0 con 0x7fd2f4106540 2024-02-14T17:40:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.850+0000 7fd2f3fff700 1 --2- 172.21.15.100:0/2016016186 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd2e4038f30 0x7fd2e403b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.850+0000 7fd2f3fff700 1 --2- 172.21.15.100:0/2016016186 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd2e4038f30 0x7fd2e403b3f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0006fd0 tx=0x7fd2e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:23.855+0000 7fd2f1ffb700 1 -- 172.21.15.100:0/2016016186 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd2e800b3e0 con 0x7fd2f4106540 2024-02-14T17:40:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.057+0000 7fd2fafc3700 1 -- 172.21.15.100:0/2016016186 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fd2f4109c50 con 0x7fd2f4106540 2024-02-14T17:40:24.955 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.058+0000 7fd2f1ffb700 1 -- 172.21.15.100:0/2016016186 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fd2e8018350 con 0x7fd2f4106540 2024-02-14T17:40:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.060+0000 7fd2fafc3700 1 -- 172.21.15.100:0/2016016186 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd2e4038f30 msgr2=0x7fd2e403b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.060+0000 7fd2fafc3700 1 --2- 172.21.15.100:0/2016016186 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd2e4038f30 0x7fd2e403b3f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0006fd0 tx=0x7fd2e0006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.060+0000 7fd2fafc3700 1 -- 172.21.15.100:0/2016016186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 msgr2=0x7fd2f41996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.060+0000 7fd2fafc3700 1 --2- 172.21.15.100:0/2016016186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f41996e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8003ec0 tx=0x7fd2e8011980 comp rx=0 tx=0).stop 2024-02-14T17:40:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.060+0000 7fd2fafc3700 1 -- 172.21.15.100:0/2016016186 shutdown_connections 2024-02-14T17:40:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.060+0000 7fd2fafc3700 1 --2- 172.21.15.100:0/2016016186 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd2e4038f30 0x7fd2e403b3f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:24.956 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.060+0000 7fd2fafc3700 1 --2- 172.21.15.100:0/2016016186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd2f4106540 0x7fd2f41996e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.060+0000 7fd2fafc3700 1 -- 172.21.15.100:0/2016016186 >> 172.21.15.100:0/2016016186 conn(0x7fd2f40fff50 msgr2=0x7fd2f4100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.060+0000 7fd2fafc3700 1 -- 172.21.15.100:0/2016016186 shutdown_connections 2024-02-14T17:40:24.957 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:24.060+0000 7fd2fafc3700 1 -- 172.21.15.100:0/2016016186 wait complete. 2024-02-14T17:40:24.958 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-14T17:40:24.958 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout disabled 2024-02-14T17:40:24.959 INFO:teuthology.orchestra.run.smithi100.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-14T17:40:24.959 INFO:teuthology.orchestra.run.smithi100.stdout:systemctl: stdout inactive 2024-02-14T17:40:24.959 INFO:teuthology.orchestra.run.smithi100.stdout:firewalld.service is not enabled 2024-02-14T17:40:24.959 INFO:teuthology.orchestra.run.smithi100.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-14T17:40:24.961 INFO:teuthology.orchestra.run.smithi100.stdout:Ceph Dashboard is now available at: 2024-02-14T17:40:24.961 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:24.961 INFO:teuthology.orchestra.run.smithi100.stdout: URL: https://smithi100.front.sepia.ceph.com:8443/ 2024-02-14T17:40:24.961 INFO:teuthology.orchestra.run.smithi100.stdout: User: admin 2024-02-14T17:40:24.961 INFO:teuthology.orchestra.run.smithi100.stdout: Password: gmmm11uytg 2024-02-14T17:40:24.961 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:24.962 INFO:teuthology.orchestra.run.smithi100.stdout:Saving cluster configuration to /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config directory 2024-02-14T17:40:24.962 INFO:teuthology.orchestra.run.smithi100.stdout:Enabling autotune for osd_memory_target 2024-02-14T17:40:25.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:25 smithi100 ceph-mon[30138]: mgrmap e13: y(active, since 6s) 2024-02-14T17:40:26.258 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.441+0000 7f923cdf7700 1 Processor -- start 2024-02-14T17:40:26.258 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.441+0000 7f923cdf7700 1 -- start start 2024-02-14T17:40:26.258 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.441+0000 7f923cdf7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238104d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.441+0000 7f923cdf7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238105320 con 0x7f9238104930 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.442+0000 7f923659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238104d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.442+0000 7f923659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238104d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51426/0 (socket says 172.21.15.100:51426) 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.442+0000 7f923659c700 1 -- 172.21.15.100:0/2979896373 learned_addr learned my addr 172.21.15.100:0/2979896373 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.442+0000 7f923659c700 1 -- 172.21.15.100:0/2979896373 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9238105b30 con 0x7f9238104930 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.442+0000 7f923659c700 1 --2- 172.21.15.100:0/2979896373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238104d50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f92280098a0 tx=0x7f9228009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45153b15d560a3e server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.442+0000 7f9235d9b700 1 -- 172.21.15.100:0/2979896373 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9228004160 con 0x7f9238104930 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.442+0000 7f9235d9b700 1 -- 172.21.15.100:0/2979896373 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f92280042c0 con 0x7f9238104930 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.442+0000 7f9235d9b700 1 -- 172.21.15.100:0/2979896373 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9228011850 con 0x7f9238104930 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.443+0000 7f923cdf7700 1 -- 172.21.15.100:0/2979896373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 msgr2=0x7f9238104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.443+0000 7f923cdf7700 1 --2- 172.21.15.100:0/2979896373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238104d50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f92280098a0 tx=0x7f9228009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.443+0000 7f923cdf7700 1 -- 172.21.15.100:0/2979896373 shutdown_connections 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.443+0000 7f923cdf7700 1 --2- 172.21.15.100:0/2979896373 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238104d50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:26.259 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.443+0000 7f923cdf7700 1 -- 172.21.15.100:0/2979896373 >> 172.21.15.100:0/2979896373 conn(0x7f92380ffef0 msgr2=0x7f9238102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.443+0000 7f923cdf7700 1 -- 172.21.15.100:0/2979896373 shutdown_connections 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.443+0000 7f923cdf7700 1 -- 172.21.15.100:0/2979896373 wait complete. 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923cdf7700 1 Processor -- start 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923cdf7700 1 -- start start 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923cdf7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238079320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923cdf7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923819c390 con 0x7f9238104930 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238079320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923659c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238079320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51438/0 (socket says 172.21.15.100:51438) 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923659c700 1 -- 172.21.15.100:0/1635652288 learned_addr learned my addr 172.21.15.100:0/1635652288 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923659c700 1 -- 172.21.15.100:0/1635652288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9228009550 con 0x7f9238104930 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923659c700 1 --2- 172.21.15.100:0/1635652288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238079320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9228006af0 tx=0x7f9228011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f922f7fe700 1 -- 172.21.15.100:0/1635652288 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92280036a0 con 0x7f9238104930 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f922f7fe700 1 -- 172.21.15.100:0/1635652288 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f922801a430 con 0x7f9238104930 2024-02-14T17:40:26.260 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923cdf7700 1 -- 172.21.15.100:0/1635652288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923819c4d0 con 0x7f9238104930 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f922f7fe700 1 -- 172.21.15.100:0/1635652288 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92280193c0 con 0x7f9238104930 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.444+0000 7f923cdf7700 1 -- 172.21.15.100:0/1635652288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923819c8b0 con 0x7f9238104930 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.445+0000 7f922f7fe700 1 -- 172.21.15.100:0/1635652288 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f9228019630 con 0x7f9238104930 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.445+0000 7f922f7fe700 1 --2- 172.21.15.100:0/1635652288 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9224038f80 0x7f922403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.445+0000 7f922f7fe700 1 -- 172.21.15.100:0/1635652288 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f922804e010 con 0x7f9238104930 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.445+0000 7f922ffff700 1 --2- 172.21.15.100:0/1635652288 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9224038f80 0x7f922403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.446+0000 7f923cdf7700 1 -- 172.21.15.100:0/1635652288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92380632a0 con 0x7f9238104930 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.446+0000 7f922ffff700 1 --2- 172.21.15.100:0/1635652288 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9224038f80 0x7f922403b440 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9220006fd0 tx=0x7f9220006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.451+0000 7f922f7fe700 1 -- 172.21.15.100:0/1635652288 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92280199d0 con 0x7f9238104930 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.645+0000 7f923cdf7700 1 -- 172.21.15.100:0/1635652288 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f923819cb60 con 0x7f9238104930 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.645+0000 7f922f7fe700 1 -- 172.21.15.100:0/1635652288 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f92280350c0 con 0x7f9238104930 2024-02-14T17:40:26.261 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.647+0000 7f923cdf7700 1 -- 172.21.15.100:0/1635652288 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9224038f80 msgr2=0x7f922403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:26.262 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.647+0000 7f923cdf7700 1 --2- 172.21.15.100:0/1635652288 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9224038f80 0x7f922403b440 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9220006fd0 tx=0x7f9220006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:26.262 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.647+0000 7f923cdf7700 1 -- 172.21.15.100:0/1635652288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 msgr2=0x7f9238079320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:26.262 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.647+0000 7f923cdf7700 1 --2- 172.21.15.100:0/1635652288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238079320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9228006af0 tx=0x7f9228011a50 comp rx=0 tx=0).stop 2024-02-14T17:40:26.262 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.647+0000 7f923cdf7700 1 -- 172.21.15.100:0/1635652288 shutdown_connections 2024-02-14T17:40:26.262 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.647+0000 7f923cdf7700 1 --2- 172.21.15.100:0/1635652288 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9224038f80 0x7f922403b440 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:26.262 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.647+0000 7f923cdf7700 1 --2- 172.21.15.100:0/1635652288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9238104930 0x7f9238079320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:26.262 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.647+0000 7f923cdf7700 1 -- 172.21.15.100:0/1635652288 >> 172.21.15.100:0/1635652288 conn(0x7f92380ffef0 msgr2=0x7f9238100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:26.262 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.647+0000 7f923cdf7700 1 -- 172.21.15.100:0/1635652288 shutdown_connections 2024-02-14T17:40:26.262 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:25.647+0000 7f923cdf7700 1 -- 172.21.15.100:0/1635652288 wait complete. 2024-02-14T17:40:27.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.770+0000 7f04a9975700 1 Processor -- start 2024-02-14T17:40:27.731 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.770+0000 7f04a9975700 1 -- start start 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.770+0000 7f04a9975700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a4104d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.770+0000 7f04a9975700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a4105320 con 0x7f04a4104930 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.770+0000 7f04a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a4104d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.770+0000 7f04a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a4104d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51450/0 (socket says 172.21.15.100:51450) 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.770+0000 7f04a2ffd700 1 -- 172.21.15.100:0/3330211785 learned_addr learned my addr 172.21.15.100:0/3330211785 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.771+0000 7f04a2ffd700 1 -- 172.21.15.100:0/3330211785 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a4105b30 con 0x7f04a4104930 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.771+0000 7f04a2ffd700 1 --2- 172.21.15.100:0/3330211785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a4104d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04940098a0 tx=0x7f0494009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bab2a3d962e1a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.771+0000 7f04a1ffb700 1 -- 172.21.15.100:0/3330211785 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0494004160 con 0x7f04a4104930 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.771+0000 7f04a1ffb700 1 -- 172.21.15.100:0/3330211785 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f04940042c0 con 0x7f04a4104930 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.771+0000 7f04a1ffb700 1 -- 172.21.15.100:0/3330211785 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0494011850 con 0x7f04a4104930 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.771+0000 7f04a9975700 1 -- 172.21.15.100:0/3330211785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 msgr2=0x7f04a4104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:27.732 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.771+0000 7f04a9975700 1 --2- 172.21.15.100:0/3330211785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a4104d50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04940098a0 tx=0x7f0494009bb0 comp rx=0 tx=0).stop 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.771+0000 7f04a9975700 1 -- 172.21.15.100:0/3330211785 shutdown_connections 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.771+0000 7f04a9975700 1 --2- 172.21.15.100:0/3330211785 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a4104d50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.771+0000 7f04a9975700 1 -- 172.21.15.100:0/3330211785 >> 172.21.15.100:0/3330211785 conn(0x7f04a40ffef0 msgr2=0x7f04a4102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.772+0000 7f04a9975700 1 -- 172.21.15.100:0/3330211785 shutdown_connections 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.772+0000 7f04a9975700 1 -- 172.21.15.100:0/3330211785 wait complete. 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.772+0000 7f04a9975700 1 Processor -- start 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.772+0000 7f04a9975700 1 -- start start 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.772+0000 7f04a9975700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.772+0000 7f04a9975700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a419bdf0 con 0x7f04a4104930 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.772+0000 7f04a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.773+0000 7f04a2ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a419b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51458/0 (socket says 172.21.15.100:51458) 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.773+0000 7f04a2ffd700 1 -- 172.21.15.100:0/359967753 learned_addr learned my addr 172.21.15.100:0/359967753 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.773+0000 7f04a2ffd700 1 -- 172.21.15.100:0/359967753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0494009550 con 0x7f04a4104930 2024-02-14T17:40:27.733 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.773+0000 7f04a2ffd700 1 --2- 172.21.15.100:0/359967753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a419b8b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0494003d50 tx=0x7f0494011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.773+0000 7f04a8973700 1 -- 172.21.15.100:0/359967753 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04940037d0 con 0x7f04a4104930 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.773+0000 7f04a8973700 1 -- 172.21.15.100:0/359967753 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0494003930 con 0x7f04a4104930 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.773+0000 7f04a8973700 1 -- 172.21.15.100:0/359967753 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04940193c0 con 0x7f04a4104930 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.773+0000 7f04a9975700 1 -- 172.21.15.100:0/359967753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a419bff0 con 0x7f04a4104930 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.773+0000 7f04a9975700 1 -- 172.21.15.100:0/359967753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a419c490 con 0x7f04a4104930 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.774+0000 7f04a8973700 1 -- 172.21.15.100:0/359967753 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f0494019630 con 0x7f04a4104930 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.774+0000 7f04a9975700 1 -- 172.21.15.100:0/359967753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a40632a0 con 0x7f04a4104930 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.774+0000 7f04a8973700 1 --2- 172.21.15.100:0/359967753 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f048c039020 0x7f048c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.774+0000 7f04a8973700 1 -- 172.21.15.100:0/359967753 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f049404d280 con 0x7f04a4104930 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.774+0000 7f04a27fc700 1 --2- 172.21.15.100:0/359967753 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f048c039020 0x7f048c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.775+0000 7f04a27fc700 1 --2- 172.21.15.100:0/359967753 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f048c039020 0x7f048c03b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0498006fd0 tx=0x7f0498006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:26.779+0000 7f04a8973700 1 -- 172.21.15.100:0/359967753 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0494010920 con 0x7f04a4104930 2024-02-14T17:40:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.075+0000 7f04a9975700 1 -- 172.21.15.100:0/359967753 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f04a419ee50 con 0x7f04a4104930 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.078+0000 7f04a8973700 1 -- 172.21.15.100:0/359967753 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f0494010380 con 0x7f04a4104930 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.080+0000 7f04a9975700 1 -- 172.21.15.100:0/359967753 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f048c039020 msgr2=0x7f048c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.080+0000 7f04a9975700 1 --2- 172.21.15.100:0/359967753 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f048c039020 0x7f048c03b4e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0498006fd0 tx=0x7f0498006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.080+0000 7f04a9975700 1 -- 172.21.15.100:0/359967753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 msgr2=0x7f04a419b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.080+0000 7f04a9975700 1 --2- 172.21.15.100:0/359967753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a419b8b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0494003d50 tx=0x7f0494011ac0 comp rx=0 tx=0).stop 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.080+0000 7f04a9975700 1 -- 172.21.15.100:0/359967753 shutdown_connections 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.080+0000 7f04a9975700 1 --2- 172.21.15.100:0/359967753 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f048c039020 0x7f048c03b4e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.080+0000 7f04a9975700 1 --2- 172.21.15.100:0/359967753 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04a4104930 0x7f04a419b8b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.080+0000 7f04a9975700 1 -- 172.21.15.100:0/359967753 >> 172.21.15.100:0/359967753 conn(0x7f04a40ffef0 msgr2=0x7f04a4100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.080+0000 7f04a9975700 1 -- 172.21.15.100:0/359967753 shutdown_connections 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr 2024-02-14T17:40:27.080+0000 7f04a9975700 1 -- 172.21.15.100:0/359967753 wait complete. 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:27.735 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout:Or, if you are only running a single cluster on this host: 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout: ceph telemetry on 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout:For more information see: 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:27.736 INFO:teuthology.orchestra.run.smithi100.stdout:Bootstrap complete. 2024-02-14T17:40:27.771 INFO:tasks.cephadm:Fetching config... 2024-02-14T17:40:27.771 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:40:27.772 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-14T17:40:27.788 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-14T17:40:27.788 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:40:27.788 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-14T17:40:27.842 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-14T17:40:27.843 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:40:27.843 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-14T17:40:27.910 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-14T17:40:27.910 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:40:27.910 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-14T17:40:27.964 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-14T17:40:27.965 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmZFIESaVkU2Xvrkbznvx6UNUzPJCbAqX4naRSa7PcLSYoY7JMGugN7P6niRwe2gzkwxL1rP8S48/fN7QzSKqNFqBSG5Yc2ADny7NxcGtpgyM4Sg+qkQWdRMuZF4Bkq8ooXXMr024ogmw7i/k8vGTTpmaXlHU8F27x9+By3EBtheMX4bUxsiilIW8qOt1N7sCHOd207eoakdfHSTetrwTYvM2BSzeSP42tRGaZieHpU+urInLy+oELI7SdDeV4Z712iSglhtdkKM+ycd/J9o5IkMFQkuVVBB6G/ErtEo93MWKGseIWgQt4NYvjWTEPTJTmC5va1GMVMmAUCFHMBW6qNmr23ksfFyi5LYThyyKSeVHGyqW7V+ZFEG9aWQTzP4rotTXg+xvNf29cl18H9W9HYXMUPZ+et9b8iwSmZ7TGymtoqzfWkdEskqHBQhDtbqGGlk1gfPjjarSB3j0tRalOfsagyWS7hXOFI6ei51OsfVE62gWdHk1GxOfb+xN0RJ8= ceph-d69a9998-cb5f-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-14T17:40:28.043 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmZFIESaVkU2Xvrkbznvx6UNUzPJCbAqX4naRSa7PcLSYoY7JMGugN7P6niRwe2gzkwxL1rP8S48/fN7QzSKqNFqBSG5Yc2ADny7NxcGtpgyM4Sg+qkQWdRMuZF4Bkq8ooXXMr024ogmw7i/k8vGTTpmaXlHU8F27x9+By3EBtheMX4bUxsiilIW8qOt1N7sCHOd207eoakdfHSTetrwTYvM2BSzeSP42tRGaZieHpU+urInLy+oELI7SdDeV4Z712iSglhtdkKM+ycd/J9o5IkMFQkuVVBB6G/ErtEo93MWKGseIWgQt4NYvjWTEPTJTmC5va1GMVMmAUCFHMBW6qNmr23ksfFyi5LYThyyKSeVHGyqW7V+ZFEG9aWQTzP4rotTXg+xvNf29cl18H9W9HYXMUPZ+et9b8iwSmZ7TGymtoqzfWkdEskqHBQhDtbqGGlk1gfPjjarSB3j0tRalOfsagyWS7hXOFI6ei51OsfVE62gWdHk1GxOfb+xN0RJ8= ceph-d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:40:28.057 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmZFIESaVkU2Xvrkbznvx6UNUzPJCbAqX4naRSa7PcLSYoY7JMGugN7P6niRwe2gzkwxL1rP8S48/fN7QzSKqNFqBSG5Yc2ADny7NxcGtpgyM4Sg+qkQWdRMuZF4Bkq8ooXXMr024ogmw7i/k8vGTTpmaXlHU8F27x9+By3EBtheMX4bUxsiilIW8qOt1N7sCHOd207eoakdfHSTetrwTYvM2BSzeSP42tRGaZieHpU+urInLy+oELI7SdDeV4Z712iSglhtdkKM+ycd/J9o5IkMFQkuVVBB6G/ErtEo93MWKGseIWgQt4NYvjWTEPTJTmC5va1GMVMmAUCFHMBW6qNmr23ksfFyi5LYThyyKSeVHGyqW7V+ZFEG9aWQTzP4rotTXg+xvNf29cl18H9W9HYXMUPZ+et9b8iwSmZ7TGymtoqzfWkdEskqHBQhDtbqGGlk1gfPjjarSB3j0tRalOfsagyWS7hXOFI6ei51OsfVE62gWdHk1GxOfb+xN0RJ8= ceph-d69a9998-cb5f-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-14T17:40:28.106 INFO:teuthology.orchestra.run.smithi179.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCmZFIESaVkU2Xvrkbznvx6UNUzPJCbAqX4naRSa7PcLSYoY7JMGugN7P6niRwe2gzkwxL1rP8S48/fN7QzSKqNFqBSG5Yc2ADny7NxcGtpgyM4Sg+qkQWdRMuZF4Bkq8ooXXMr024ogmw7i/k8vGTTpmaXlHU8F27x9+By3EBtheMX4bUxsiilIW8qOt1N7sCHOd207eoakdfHSTetrwTYvM2BSzeSP42tRGaZieHpU+urInLy+oELI7SdDeV4Z712iSglhtdkKM+ycd/J9o5IkMFQkuVVBB6G/ErtEo93MWKGseIWgQt4NYvjWTEPTJTmC5va1GMVMmAUCFHMBW6qNmr23ksfFyi5LYThyyKSeVHGyqW7V+ZFEG9aWQTzP4rotTXg+xvNf29cl18H9W9HYXMUPZ+et9b8iwSmZ7TGymtoqzfWkdEskqHBQhDtbqGGlk1gfPjjarSB3j0tRalOfsagyWS7hXOFI6ei51OsfVE62gWdHk1GxOfb+xN0RJ8= ceph-d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:40:28.121 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-14T17:40:28.335 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/359967753' entity='client.admin' 2024-02-14T17:40:28.541 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:40:30.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.943+0000 7f3ca87a8700 1 -- 172.21.15.100:0/2428546093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca01005d0 msgr2=0x7f3ca01009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:30.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.943+0000 7f3ca87a8700 1 --2- 172.21.15.100:0/2428546093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca01005d0 0x7f3ca01009f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009910 tx=0x7f3c90009c20 comp rx=0 tx=0).stop 2024-02-14T17:40:30.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.943+0000 7f3ca87a8700 1 -- 172.21.15.100:0/2428546093 shutdown_connections 2024-02-14T17:40:30.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.943+0000 7f3ca87a8700 1 --2- 172.21.15.100:0/2428546093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca01005d0 0x7f3ca01009f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:30.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.943+0000 7f3ca87a8700 1 -- 172.21.15.100:0/2428546093 >> 172.21.15.100:0/2428546093 conn(0x7f3ca00fbbb0 msgr2=0x7f3ca00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:30.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.943+0000 7f3ca87a8700 1 -- 172.21.15.100:0/2428546093 shutdown_connections 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.943+0000 7f3ca87a8700 1 -- 172.21.15.100:0/2428546093 wait complete. 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.944+0000 7f3ca87a8700 1 Processor -- start 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.944+0000 7f3ca87a8700 1 -- start start 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.944+0000 7f3ca87a8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca01005d0 0x7f3ca006a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.944+0000 7f3ca87a8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca006aa00 con 0x7f3ca01005d0 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.944+0000 7f3ca6544700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca01005d0 0x7f3ca006a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.944+0000 7f3ca6544700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca01005d0 0x7f3ca006a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51482/0 (socket says 172.21.15.100:51482) 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.944+0000 7f3ca6544700 1 -- 172.21.15.100:0/1335964692 learned_addr learned my addr 172.21.15.100:0/1335964692 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.945+0000 7f3ca6544700 1 -- 172.21.15.100:0/1335964692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900095f0 con 0x7f3ca01005d0 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.945+0000 7f3ca6544700 1 --2- 172.21.15.100:0/1335964692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca01005d0 0x7f3ca006a4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c90005660 tx=0x7f3c90005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.945+0000 7f3c977fe700 1 -- 172.21.15.100:0/1335964692 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c9000b960 con 0x7f3ca01005d0 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.945+0000 7f3c977fe700 1 -- 172.21.15.100:0/1335964692 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3c9000f460 con 0x7f3ca01005d0 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.945+0000 7f3ca87a8700 1 -- 172.21.15.100:0/1335964692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca0196ce0 con 0x7f3ca01005d0 2024-02-14T17:40:30.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.945+0000 7f3c977fe700 1 -- 172.21.15.100:0/1335964692 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c90017440 con 0x7f3ca01005d0 2024-02-14T17:40:30.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.945+0000 7f3ca87a8700 1 -- 172.21.15.100:0/1335964692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca019b5a0 con 0x7f3ca01005d0 2024-02-14T17:40:30.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.946+0000 7f3c977fe700 1 -- 172.21.15.100:0/1335964692 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f3c9000bac0 con 0x7f3ca01005d0 2024-02-14T17:40:30.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.946+0000 7f3ca87a8700 1 -- 172.21.15.100:0/1335964692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca018ee50 con 0x7f3ca01005d0 2024-02-14T17:40:30.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.946+0000 7f3c977fe700 1 --2- 172.21.15.100:0/1335964692 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f3c8c038fe0 0x7f3c8c03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:30.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.946+0000 7f3c977fe700 1 -- 172.21.15.100:0/1335964692 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3c9004d130 con 0x7f3ca01005d0 2024-02-14T17:40:30.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.946+0000 7f3ca5d43700 1 --2- 172.21.15.100:0/1335964692 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f3c8c038fe0 0x7f3c8c03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:30.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.947+0000 7f3ca5d43700 1 --2- 172.21.15.100:0/1335964692 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f3c8c038fe0 0x7f3c8c03b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c006fd0 tx=0x7f3c9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:30.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:30.951+0000 7f3c977fe700 1 -- 172.21.15.100:0/1335964692 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3c9000f8b0 con 0x7f3ca01005d0 2024-02-14T17:40:31.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.142+0000 7f3ca87a8700 1 -- 172.21.15.100:0/1335964692 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f3ca01058d0 con 0x7f3ca01005d0 2024-02-14T17:40:31.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.144+0000 7f3c977fe700 1 -- 172.21.15.100:0/1335964692 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3c90027030 con 0x7f3ca01005d0 2024-02-14T17:40:31.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.146+0000 7f3ca87a8700 1 -- 172.21.15.100:0/1335964692 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f3c8c038fe0 msgr2=0x7f3c8c03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:31.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.146+0000 7f3ca87a8700 1 --2- 172.21.15.100:0/1335964692 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f3c8c038fe0 0x7f3c8c03b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c006fd0 tx=0x7f3c9c006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:31.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.146+0000 7f3ca87a8700 1 -- 172.21.15.100:0/1335964692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca01005d0 msgr2=0x7f3ca006a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:31.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.146+0000 7f3ca87a8700 1 --2- 172.21.15.100:0/1335964692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca01005d0 0x7f3ca006a4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c90005660 tx=0x7f3c90005740 comp rx=0 tx=0).stop 2024-02-14T17:40:31.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.146+0000 7f3ca87a8700 1 -- 172.21.15.100:0/1335964692 shutdown_connections 2024-02-14T17:40:31.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.146+0000 7f3ca87a8700 1 --2- 172.21.15.100:0/1335964692 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f3c8c038fe0 0x7f3c8c03b4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:31.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.146+0000 7f3ca87a8700 1 --2- 172.21.15.100:0/1335964692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3ca01005d0 0x7f3ca006a4c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:31.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.146+0000 7f3ca87a8700 1 -- 172.21.15.100:0/1335964692 >> 172.21.15.100:0/1335964692 conn(0x7f3ca00fbbb0 msgr2=0x7f3ca00fc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:31.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.146+0000 7f3ca87a8700 1 -- 172.21.15.100:0/1335964692 shutdown_connections 2024-02-14T17:40:31.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:31.146+0000 7f3ca87a8700 1 -- 172.21.15.100:0/1335964692 wait complete. 2024-02-14T17:40:31.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:31.785 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:31.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:31.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:31.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:31.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:31.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1335964692' entity='client.admin' 2024-02-14T17:40:31.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:31.786 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:31.813 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-14T17:40:31.813 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-14T17:40:32.049 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:40:33.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.681+0000 7fe0f907e700 1 -- 172.21.15.100:0/3037015913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4100610 msgr2=0x7fe0f4100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.681+0000 7fe0f907e700 1 --2- 172.21.15.100:0/3037015913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4100610 0x7fe0f4100a30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400b0a0 tx=0x7fe0e400b3b0 comp rx=0 tx=0).stop 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.681+0000 7fe0f907e700 1 -- 172.21.15.100:0/3037015913 shutdown_connections 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.681+0000 7fe0f907e700 1 --2- 172.21.15.100:0/3037015913 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4100610 0x7fe0f4100a30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.681+0000 7fe0f907e700 1 -- 172.21.15.100:0/3037015913 >> 172.21.15.100:0/3037015913 conn(0x7fe0f40fbbd0 msgr2=0x7fe0f40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.681+0000 7fe0f907e700 1 -- 172.21.15.100:0/3037015913 shutdown_connections 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.681+0000 7fe0f907e700 1 -- 172.21.15.100:0/3037015913 wait complete. 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.681+0000 7fe0f907e700 1 Processor -- start 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.682+0000 7fe0f907e700 1 -- start start 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.682+0000 7fe0f907e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4100610 0x7fe0f4069af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.682+0000 7fe0f907e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f406a030 con 0x7fe0f4100610 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.682+0000 7fe0f2d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4100610 0x7fe0f4069af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:33.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.682+0000 7fe0f2d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4100610 0x7fe0f4069af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35698/0 (socket says 172.21.15.100:35698) 2024-02-14T17:40:33.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.682+0000 7fe0f2d9d700 1 -- 172.21.15.100:0/746261641 learned_addr learned my addr 172.21.15.100:0/746261641 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:33.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.682+0000 7fe0f2d9d700 1 -- 172.21.15.100:0/746261641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e4009d00 con 0x7fe0f4100610 2024-02-14T17:40:33.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.682+0000 7fe0f2d9d700 1 --2- 172.21.15.100:0/746261641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4100610 0x7fe0f4069af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4015040 tx=0x7fe0e4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:33.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.683+0000 7fe0ebfff700 1 -- 172.21.15.100:0/746261641 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0e400e040 con 0x7fe0f4100610 2024-02-14T17:40:33.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.683+0000 7fe0ebfff700 1 -- 172.21.15.100:0/746261641 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe0e400be60 con 0x7fe0f4100610 2024-02-14T17:40:33.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.683+0000 7fe0f907e700 1 -- 172.21.15.100:0/746261641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f4196c90 con 0x7fe0f4100610 2024-02-14T17:40:33.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.683+0000 7fe0ebfff700 1 -- 172.21.15.100:0/746261641 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0e40125c0 con 0x7fe0f4100610 2024-02-14T17:40:33.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.683+0000 7fe0f907e700 1 -- 172.21.15.100:0/746261641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f419b550 con 0x7fe0f4100610 2024-02-14T17:40:33.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.683+0000 7fe0ebfff700 1 -- 172.21.15.100:0/746261641 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7fe0e4019070 con 0x7fe0f4100610 2024-02-14T17:40:33.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.683+0000 7fe0f907e700 1 -- 172.21.15.100:0/746261641 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f418ee20 con 0x7fe0f4100610 2024-02-14T17:40:33.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.684+0000 7fe0ebfff700 1 --2- 172.21.15.100:0/746261641 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe0e0038f80 0x7fe0e003b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:33.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.684+0000 7fe0ebfff700 1 -- 172.21.15.100:0/746261641 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe0e404d490 con 0x7fe0f4100610 2024-02-14T17:40:33.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.684+0000 7fe0f259c700 1 --2- 172.21.15.100:0/746261641 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe0e0038f80 0x7fe0e003b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:33.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.684+0000 7fe0f259c700 1 --2- 172.21.15.100:0/746261641 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe0e0038f80 0x7fe0e003b440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc006fd0 tx=0x7fe0dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:33.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.689+0000 7fe0ebfff700 1 -- 172.21.15.100:0/746261641 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0e401a380 con 0x7fe0f4100610 2024-02-14T17:40:33.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.886+0000 7fe0f907e700 1 -- 172.21.15.100:0/746261641 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0f4105b40 con 0x7fe0e0038f80 2024-02-14T17:40:33.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.890+0000 7fe0ebfff700 1 -- 172.21.15.100:0/746261641 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe0f4105b40 con 0x7fe0e0038f80 2024-02-14T17:40:33.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.892+0000 7fe0f907e700 1 -- 172.21.15.100:0/746261641 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe0e0038f80 msgr2=0x7fe0e003b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:33.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.892+0000 7fe0f907e700 1 --2- 172.21.15.100:0/746261641 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe0e0038f80 0x7fe0e003b440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc006fd0 tx=0x7fe0dc006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:33.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.892+0000 7fe0f907e700 1 -- 172.21.15.100:0/746261641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4100610 msgr2=0x7fe0f4069af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:33.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.892+0000 7fe0f907e700 1 --2- 172.21.15.100:0/746261641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4100610 0x7fe0f4069af0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4015040 tx=0x7fe0e4003ce0 comp rx=0 tx=0).stop 2024-02-14T17:40:33.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.892+0000 7fe0f907e700 1 -- 172.21.15.100:0/746261641 shutdown_connections 2024-02-14T17:40:33.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.892+0000 7fe0f907e700 1 --2- 172.21.15.100:0/746261641 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe0e0038f80 0x7fe0e003b440 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:33.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.892+0000 7fe0f907e700 1 --2- 172.21.15.100:0/746261641 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe0f4100610 0x7fe0f4069af0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:33.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.892+0000 7fe0f907e700 1 -- 172.21.15.100:0/746261641 >> 172.21.15.100:0/746261641 conn(0x7fe0f40fbbd0 msgr2=0x7fe0f40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:33.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.892+0000 7fe0f907e700 1 -- 172.21.15.100:0/746261641 shutdown_connections 2024-02-14T17:40:33.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:33.892+0000 7fe0f907e700 1 -- 172.21.15.100:0/746261641 wait complete. 2024-02-14T17:40:34.671 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi179 2024-02-14T17:40:34.671 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:40:34.671 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.conf 2024-02-14T17:40:34.690 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:40:34.690 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:40:34.746 INFO:tasks.cephadm:Adding host smithi179 to orchestrator... 2024-02-14T17:40:34.747 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch host add smithi179 2024-02-14T17:40:34.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:34 smithi100 ceph-mon[30138]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:34.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:34.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:34.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:34 smithi100 ceph-mon[30138]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-14T17:40:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:34 smithi100 ceph-mon[30138]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:40:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:34.983 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:40:36.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:35 smithi100 ceph-mon[30138]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:40:36.107 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:35 smithi100 ceph-mon[30138]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:40:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.687+0000 7f9fb4055700 1 -- 172.21.15.100:0/4197626016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fac0ffef0 msgr2=0x7f9fac1002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.687+0000 7f9fb4055700 1 --2- 172.21.15.100:0/4197626016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fac0ffef0 0x7f9fac1002d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009910 tx=0x7f9fa8009c20 comp rx=0 tx=0).stop 2024-02-14T17:40:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.687+0000 7f9fb4055700 1 -- 172.21.15.100:0/4197626016 shutdown_connections 2024-02-14T17:40:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.687+0000 7f9fb4055700 1 --2- 172.21.15.100:0/4197626016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fac0ffef0 0x7f9fac1002d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.687+0000 7f9fb4055700 1 -- 172.21.15.100:0/4197626016 >> 172.21.15.100:0/4197626016 conn(0x7f9fac0fb7e0 msgr2=0x7f9fac0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.687+0000 7f9fb4055700 1 -- 172.21.15.100:0/4197626016 shutdown_connections 2024-02-14T17:40:36.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.687+0000 7f9fb4055700 1 -- 172.21.15.100:0/4197626016 wait complete. 2024-02-14T17:40:36.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.688+0000 7f9fb4055700 1 Processor -- start 2024-02-14T17:40:36.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.688+0000 7f9fb4055700 1 -- start start 2024-02-14T17:40:36.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.688+0000 7f9fb4055700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fac0ffef0 0x7f9fac196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:36.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.688+0000 7f9fb4055700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac196ad0 con 0x7f9fac0ffef0 2024-02-14T17:40:36.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.688+0000 7f9fb1df1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fac0ffef0 0x7f9fac196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:36.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.688+0000 7f9fb1df1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fac0ffef0 0x7f9fac196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35718/0 (socket says 172.21.15.100:35718) 2024-02-14T17:40:36.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.688+0000 7f9fb1df1700 1 -- 172.21.15.100:0/2114619516 learned_addr learned my addr 172.21.15.100:0/2114619516 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:36.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.688+0000 7f9fb1df1700 1 -- 172.21.15.100:0/2114619516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80095f0 con 0x7f9fac0ffef0 2024-02-14T17:40:36.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.689+0000 7f9fb1df1700 1 --2- 172.21.15.100:0/2114619516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fac0ffef0 0x7f9fac196590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8006010 tx=0x7f9fa8005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:36.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.689+0000 7f9fa2ffd700 1 -- 172.21.15.100:0/2114619516 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fa801d070 con 0x7f9fac0ffef0 2024-02-14T17:40:36.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.689+0000 7f9fa2ffd700 1 -- 172.21.15.100:0/2114619516 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f9fa800b750 con 0x7f9fac0ffef0 2024-02-14T17:40:36.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.689+0000 7f9fb4055700 1 -- 172.21.15.100:0/2114619516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fac196cd0 con 0x7f9fac0ffef0 2024-02-14T17:40:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.689+0000 7f9fa2ffd700 1 -- 172.21.15.100:0/2114619516 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fa8003810 con 0x7f9fac0ffef0 2024-02-14T17:40:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.689+0000 7f9fb4055700 1 -- 172.21.15.100:0/2114619516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fac1970f0 con 0x7f9fac0ffef0 2024-02-14T17:40:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.690+0000 7f9fa2ffd700 1 -- 172.21.15.100:0/2114619516 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f9fa8003a50 con 0x7f9fac0ffef0 2024-02-14T17:40:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.690+0000 7f9fb4055700 1 -- 172.21.15.100:0/2114619516 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fac0632a0 con 0x7f9fac0ffef0 2024-02-14T17:40:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.690+0000 7f9fa2ffd700 1 --2- 172.21.15.100:0/2114619516 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9f98038f80 0x7f9f9803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.690+0000 7f9fa2ffd700 1 -- 172.21.15.100:0/2114619516 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9fa804ce90 con 0x7f9fac0ffef0 2024-02-14T17:40:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.690+0000 7f9fb15f0700 1 --2- 172.21.15.100:0/2114619516 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9f98038f80 0x7f9f9803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:36.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.690+0000 7f9fb15f0700 1 --2- 172.21.15.100:0/2114619516 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9f98038f80 0x7f9f9803b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c006fd0 tx=0x7f9f9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:36.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.695+0000 7f9fa2ffd700 1 -- 172.21.15.100:0/2114619516 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9fa8029360 con 0x7f9fac0ffef0 2024-02-14T17:40:36.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:36.890+0000 7f9fb4055700 1 -- 172.21.15.100:0/2114619516 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fac1095a0 con 0x7f9f98038f80 2024-02-14T17:40:37.489 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:37 smithi100 ceph-mon[30138]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:38 smithi100 ceph-mon[30138]: Deploying cephadm binary to smithi179 2024-02-14T17:40:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:38 smithi100 ceph-mon[30138]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:41.108 INFO:teuthology.orchestra.run.smithi100.stdout:Added host 'smithi179' with addr '172.21.15.179' 2024-02-14T17:40:41.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.105+0000 7f9fa2ffd700 1 -- 172.21.15.100:0/2114619516 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9fac1095a0 con 0x7f9f98038f80 2024-02-14T17:40:41.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.107+0000 7f9fb4055700 1 -- 172.21.15.100:0/2114619516 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9f98038f80 msgr2=0x7f9f9803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:41.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.107+0000 7f9fb4055700 1 --2- 172.21.15.100:0/2114619516 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9f98038f80 0x7f9f9803b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c006fd0 tx=0x7f9f9c006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:41.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.107+0000 7f9fb4055700 1 -- 172.21.15.100:0/2114619516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fac0ffef0 msgr2=0x7f9fac196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:41.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.107+0000 7f9fb4055700 1 --2- 172.21.15.100:0/2114619516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fac0ffef0 0x7f9fac196590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8006010 tx=0x7f9fa8005710 comp rx=0 tx=0).stop 2024-02-14T17:40:41.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.107+0000 7f9fb4055700 1 -- 172.21.15.100:0/2114619516 shutdown_connections 2024-02-14T17:40:41.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.107+0000 7f9fb4055700 1 --2- 172.21.15.100:0/2114619516 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f9f98038f80 0x7f9f9803b440 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:41.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.107+0000 7f9fb4055700 1 --2- 172.21.15.100:0/2114619516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9fac0ffef0 0x7f9fac196590 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:41.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.107+0000 7f9fb4055700 1 -- 172.21.15.100:0/2114619516 >> 172.21.15.100:0/2114619516 conn(0x7f9fac0fb7e0 msgr2=0x7f9fac107ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:41.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.108+0000 7f9fb4055700 1 -- 172.21.15.100:0/2114619516 shutdown_connections 2024-02-14T17:40:41.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:41.108+0000 7f9fb4055700 1 -- 172.21.15.100:0/2114619516 wait complete. 2024-02-14T17:40:41.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:41 smithi100 ceph-mon[30138]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:41.797 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch host ls --format=json 2024-02-14T17:40:42.037 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:40:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:42 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:42 smithi100 ceph-mon[30138]: Added host smithi179 2024-02-14T17:40:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:42 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:42.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:42 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:43.270 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:43 smithi100 ceph-mon[30138]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:43.270 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.763+0000 7f5504e94700 1 -- 172.21.15.100:0/3380344281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55000fff50 msgr2=0x7f5500100330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.763+0000 7f5504e94700 1 --2- 172.21.15.100:0/3380344281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55000fff50 0x7f5500100330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009910 tx=0x7f54f0009c20 comp rx=0 tx=0).stop 2024-02-14T17:40:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.763+0000 7f5504e94700 1 -- 172.21.15.100:0/3380344281 shutdown_connections 2024-02-14T17:40:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.763+0000 7f5504e94700 1 --2- 172.21.15.100:0/3380344281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55000fff50 0x7f5500100330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:43.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.763+0000 7f5504e94700 1 -- 172.21.15.100:0/3380344281 >> 172.21.15.100:0/3380344281 conn(0x7f5500077c90 msgr2=0x7f55000780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:43.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.763+0000 7f5504e94700 1 -- 172.21.15.100:0/3380344281 shutdown_connections 2024-02-14T17:40:43.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.763+0000 7f5504e94700 1 -- 172.21.15.100:0/3380344281 wait complete. 2024-02-14T17:40:43.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.764+0000 7f5504e94700 1 Processor -- start 2024-02-14T17:40:43.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.764+0000 7f5504e94700 1 -- start start 2024-02-14T17:40:43.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.764+0000 7f5504e94700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55000fff50 0x7f55001973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:43.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.764+0000 7f5504e94700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5500108a60 con 0x7f55000fff50 2024-02-14T17:40:43.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.764+0000 7f54fe59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55000fff50 0x7f55001973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:43.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.764+0000 7f54fe59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55000fff50 0x7f55001973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33446/0 (socket says 172.21.15.100:33446) 2024-02-14T17:40:43.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.764+0000 7f54fe59c700 1 -- 172.21.15.100:0/2809610775 learned_addr learned my addr 172.21.15.100:0/2809610775 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:43.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.765+0000 7f54fe59c700 1 -- 172.21.15.100:0/2809610775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f00095f0 con 0x7f55000fff50 2024-02-14T17:40:43.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.765+0000 7f54fe59c700 1 --2- 172.21.15.100:0/2809610775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55000fff50 0x7f55001973e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005660 tx=0x7f54f0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:43.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.765+0000 7f54f7fff700 1 -- 172.21.15.100:0/2809610775 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54f001d070 con 0x7f55000fff50 2024-02-14T17:40:43.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.765+0000 7f54f7fff700 1 -- 172.21.15.100:0/2809610775 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f54f000b750 con 0x7f55000fff50 2024-02-14T17:40:43.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.765+0000 7f5504e94700 1 -- 172.21.15.100:0/2809610775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5500197980 con 0x7f55000fff50 2024-02-14T17:40:43.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.765+0000 7f54f7fff700 1 -- 172.21.15.100:0/2809610775 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54f00037b0 con 0x7f55000fff50 2024-02-14T17:40:43.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.765+0000 7f5504e94700 1 -- 172.21.15.100:0/2809610775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5500197e00 con 0x7f55000fff50 2024-02-14T17:40:43.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.766+0000 7f54f7fff700 1 -- 172.21.15.100:0/2809610775 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f54f0003a70 con 0x7f55000fff50 2024-02-14T17:40:43.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.766+0000 7f54f7fff700 1 --2- 172.21.15.100:0/2809610775 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f54ec038f80 0x7f54ec03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:43.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.766+0000 7f54f7fff700 1 -- 172.21.15.100:0/2809610775 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f54f004db10 con 0x7f55000fff50 2024-02-14T17:40:43.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.766+0000 7f54f5d9b700 1 --2- 172.21.15.100:0/2809610775 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f54ec038f80 0x7f54ec03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:43.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.766+0000 7f5504e94700 1 -- 172.21.15.100:0/2809610775 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5500101420 con 0x7f55000fff50 2024-02-14T17:40:43.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.766+0000 7f54f5d9b700 1 --2- 172.21.15.100:0/2809610775 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f54ec038f80 0x7f54ec03b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f54e8006fd0 tx=0x7f54e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:43.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.771+0000 7f54f7fff700 1 -- 172.21.15.100:0/2809610775 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f54f0026020 con 0x7f55000fff50 2024-02-14T17:40:43.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.969+0000 7f5504e94700 1 -- 172.21.15.100:0/2809610775 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5500198190 con 0x7f54ec038f80 2024-02-14T17:40:43.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.970+0000 7f54f7fff700 1 -- 172.21.15.100:0/2809610775 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f5500198190 con 0x7f54ec038f80 2024-02-14T17:40:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:40:43.972 INFO:teuthology.orchestra.run.smithi100.stdout:[{"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.179", "hostname": "smithi179", "labels": [], "status": ""}] 2024-02-14T17:40:43.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.972+0000 7f5504e94700 1 -- 172.21.15.100:0/2809610775 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f54ec038f80 msgr2=0x7f54ec03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:43.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.972+0000 7f5504e94700 1 --2- 172.21.15.100:0/2809610775 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f54ec038f80 0x7f54ec03b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f54e8006fd0 tx=0x7f54e8006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:43.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.972+0000 7f5504e94700 1 -- 172.21.15.100:0/2809610775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55000fff50 msgr2=0x7f55001973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:43.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.972+0000 7f5504e94700 1 --2- 172.21.15.100:0/2809610775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55000fff50 0x7f55001973e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005660 tx=0x7f54f0005740 comp rx=0 tx=0).stop 2024-02-14T17:40:43.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.973+0000 7f5504e94700 1 -- 172.21.15.100:0/2809610775 shutdown_connections 2024-02-14T17:40:43.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.973+0000 7f5504e94700 1 --2- 172.21.15.100:0/2809610775 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f54ec038f80 0x7f54ec03b440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:43.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.973+0000 7f5504e94700 1 --2- 172.21.15.100:0/2809610775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55000fff50 0x7f55001973e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:43.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.973+0000 7f5504e94700 1 -- 172.21.15.100:0/2809610775 >> 172.21.15.100:0/2809610775 conn(0x7f5500077c90 msgr2=0x7f55000fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:43.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.973+0000 7f5504e94700 1 -- 172.21.15.100:0/2809610775 shutdown_connections 2024-02-14T17:40:43.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:43.973+0000 7f5504e94700 1 -- 172.21.15.100:0/2809610775 wait complete. 2024-02-14T17:40:44.661 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-14T17:40:44.662 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd crush tunables default 2024-02-14T17:40:44.895 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:40:44.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:44.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:44 smithi100 ceph-mon[30138]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:40:44.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:44 smithi100 ceph-mon[30138]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.506+0000 7fe1942e0700 1 -- 172.21.15.100:0/3183382989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe18c106620 msgr2=0x7fe18c106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.506+0000 7fe1942e0700 1 --2- 172.21.15.100:0/3183382989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe18c106620 0x7fe18c106a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe188009910 tx=0x7fe188009c20 comp rx=0 tx=0).stop 2024-02-14T17:40:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.507+0000 7fe1942e0700 1 -- 172.21.15.100:0/3183382989 shutdown_connections 2024-02-14T17:40:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.507+0000 7fe1942e0700 1 --2- 172.21.15.100:0/3183382989 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe18c106620 0x7fe18c106a00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.507+0000 7fe1942e0700 1 -- 172.21.15.100:0/3183382989 >> 172.21.15.100:0/3183382989 conn(0x7fe18c0fc080 msgr2=0x7fe18c0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.507+0000 7fe1942e0700 1 -- 172.21.15.100:0/3183382989 shutdown_connections 2024-02-14T17:40:46.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.507+0000 7fe1942e0700 1 -- 172.21.15.100:0/3183382989 wait complete. 2024-02-14T17:40:46.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.507+0000 7fe1942e0700 1 Processor -- start 2024-02-14T17:40:46.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.507+0000 7fe1942e0700 1 -- start start 2024-02-14T17:40:46.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe1942e0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe18c106620 0x7fe18c195320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:46.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe1942e0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c195860 con 0x7fe18c106620 2024-02-14T17:40:46.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe19207c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe18c106620 0x7fe18c195320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:46.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe19207c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe18c106620 0x7fe18c195320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33466/0 (socket says 172.21.15.100:33466) 2024-02-14T17:40:46.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe19207c700 1 -- 172.21.15.100:0/2209095686 learned_addr learned my addr 172.21.15.100:0/2209095686 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe19207c700 1 -- 172.21.15.100:0/2209095686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1880095f0 con 0x7fe18c106620 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe19207c700 1 --2- 172.21.15.100:0/2209095686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe18c106620 0x7fe18c195320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe188005660 tx=0x7fe188005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe1837fe700 1 -- 172.21.15.100:0/2209095686 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe18801f070 con 0x7fe18c106620 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe1942e0700 1 -- 172.21.15.100:0/2209095686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c195a60 con 0x7fe18c106620 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe1837fe700 1 -- 172.21.15.100:0/2209095686 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe18800b750 con 0x7fe18c106620 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe1837fe700 1 -- 172.21.15.100:0/2209095686 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1880037b0 con 0x7fe18c106620 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.508+0000 7fe1942e0700 1 -- 172.21.15.100:0/2209095686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c195f60 con 0x7fe18c106620 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.509+0000 7fe1837fe700 1 -- 172.21.15.100:0/2209095686 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7fe188004390 con 0x7fe18c106620 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.509+0000 7fe1942e0700 1 -- 172.21.15.100:0/2209095686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c0632a0 con 0x7fe18c106620 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.509+0000 7fe1837fe700 1 --2- 172.21.15.100:0/2209095686 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe178038f80 0x7fe17803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.509+0000 7fe1837fe700 1 -- 172.21.15.100:0/2209095686 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe18804efd0 con 0x7fe18c106620 2024-02-14T17:40:46.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.510+0000 7fe19187b700 1 --2- 172.21.15.100:0/2209095686 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe178038f80 0x7fe17803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:46.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.510+0000 7fe19187b700 1 --2- 172.21.15.100:0/2209095686 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe178038f80 0x7fe17803b440 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe17c006fd0 tx=0x7fe17c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:46.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.514+0000 7fe1837fe700 1 -- 172.21.15.100:0/2209095686 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe188003df0 con 0x7fe18c106620 2024-02-14T17:40:46.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:46.712+0000 7fe1942e0700 1 -- 172.21.15.100:0/2209095686 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe18c196680 con 0x7fe18c106620 2024-02-14T17:40:47.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.074+0000 7fe1837fe700 1 -- 172.21.15.100:0/2209095686 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe188028020 con 0x7fe18c106620 2024-02-14T17:40:47.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.076+0000 7fe1942e0700 1 -- 172.21.15.100:0/2209095686 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe178038f80 msgr2=0x7fe17803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:47.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.076+0000 7fe1942e0700 1 --2- 172.21.15.100:0/2209095686 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe178038f80 0x7fe17803b440 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe17c006fd0 tx=0x7fe17c006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:47.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.076+0000 7fe1942e0700 1 -- 172.21.15.100:0/2209095686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe18c106620 msgr2=0x7fe18c195320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:47.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.076+0000 7fe1942e0700 1 --2- 172.21.15.100:0/2209095686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe18c106620 0x7fe18c195320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe188005660 tx=0x7fe188005740 comp rx=0 tx=0).stop 2024-02-14T17:40:47.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.077+0000 7fe1942e0700 1 -- 172.21.15.100:0/2209095686 shutdown_connections 2024-02-14T17:40:47.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.077+0000 7fe1942e0700 1 --2- 172.21.15.100:0/2209095686 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fe178038f80 0x7fe17803b440 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:47.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.077+0000 7fe1942e0700 1 --2- 172.21.15.100:0/2209095686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe18c106620 0x7fe18c195320 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:47.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.077+0000 7fe1942e0700 1 -- 172.21.15.100:0/2209095686 >> 172.21.15.100:0/2209095686 conn(0x7fe18c0fc080 msgr2=0x7fe18c0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:47.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.077+0000 7fe1942e0700 1 -- 172.21.15.100:0/2209095686 shutdown_connections 2024-02-14T17:40:47.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:40:47.077+0000 7fe1942e0700 1 -- 172.21.15.100:0/2209095686 wait complete. 2024-02-14T17:40:47.079 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-02-14T17:40:47.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:47 smithi100 ceph-mon[30138]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:47.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209095686' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T17:40:47.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:47.782 INFO:tasks.cephadm:Adding mon.a on smithi100 2024-02-14T17:40:47.782 INFO:tasks.cephadm:Adding mon.c on smithi100 2024-02-14T17:40:47.782 INFO:tasks.cephadm:Adding mon.b on smithi179 2024-02-14T17:40:47.783 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply mon '3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi179:172.21.15.179=b' 2024-02-14T17:40:48.006 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:40:48.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:48 smithi100 ceph-mon[30138]: Updating smithi179:/etc/ceph/ceph.conf 2024-02-14T17:40:48.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:48 smithi100 ceph-mon[30138]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:40:48.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209095686' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T17:40:48.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:48 smithi100 ceph-mon[30138]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:40:48.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:48 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:48.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:48 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:48.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:48 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:49.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:49 smithi100 ceph-mon[30138]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:40:49.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:49 smithi100 ceph-mon[30138]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:40:49.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:49 smithi100 ceph-mon[30138]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:49.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.505+0000 7fa72bf1d700 1 -- 172.21.15.179:0/3960322685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7241027f0 msgr2=0x7fa724102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:49.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.505+0000 7fa72bf1d700 1 --2- 172.21.15.179:0/3960322685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7241027f0 0x7fa724102bd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009910 tx=0x7fa71c009c20 comp rx=0 tx=0).stop 2024-02-14T17:40:49.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.505+0000 7fa72bf1d700 1 -- 172.21.15.179:0/3960322685 shutdown_connections 2024-02-14T17:40:49.531 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.505+0000 7fa72bf1d700 1 --2- 172.21.15.179:0/3960322685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7241027f0 0x7fa724102bd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.505+0000 7fa72bf1d700 1 -- 172.21.15.179:0/3960322685 >> 172.21.15.179:0/3960322685 conn(0x7fa7240fc020 msgr2=0x7fa7240fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.505+0000 7fa72bf1d700 1 -- 172.21.15.179:0/3960322685 shutdown_connections 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.505+0000 7fa72bf1d700 1 -- 172.21.15.179:0/3960322685 wait complete. 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.506+0000 7fa72bf1d700 1 Processor -- start 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.506+0000 7fa72bf1d700 1 -- start start 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.506+0000 7fa72bf1d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7241027f0 0x7fa7241965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.506+0000 7fa72bf1d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa724196b20 con 0x7fa7241027f0 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.506+0000 7fa729cb9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7241027f0 0x7fa7241965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.506+0000 7fa729cb9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7241027f0 0x7fa7241965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.179:42556/0 (socket says 172.21.15.179:42556) 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.506+0000 7fa729cb9700 1 -- 172.21.15.179:0/2279863683 learned_addr learned my addr 172.21.15.179:0/2279863683 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.507+0000 7fa729cb9700 1 -- 172.21.15.179:0/2279863683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c0095f0 con 0x7fa7241027f0 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.507+0000 7fa729cb9700 1 --2- 172.21.15.179:0/2279863683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7241027f0 0x7fa7241965e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa71c006010 tx=0x7fa71c005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.507+0000 7fa722ffd700 1 -- 172.21.15.179:0/2279863683 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa71c01d070 con 0x7fa7241027f0 2024-02-14T17:40:49.532 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.507+0000 7fa72bf1d700 1 -- 172.21.15.179:0/2279863683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa724196d20 con 0x7fa7241027f0 2024-02-14T17:40:49.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.507+0000 7fa722ffd700 1 -- 172.21.15.179:0/2279863683 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa71c00b750 con 0x7fa7241027f0 2024-02-14T17:40:49.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.507+0000 7fa722ffd700 1 -- 172.21.15.179:0/2279863683 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa71c0037f0 con 0x7fa7241027f0 2024-02-14T17:40:49.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.507+0000 7fa72bf1d700 1 -- 172.21.15.179:0/2279863683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa724197140 con 0x7fa7241027f0 2024-02-14T17:40:49.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.508+0000 7fa72bf1d700 1 -- 172.21.15.179:0/2279863683 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7240632a0 con 0x7fa7241027f0 2024-02-14T17:40:49.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.508+0000 7fa722ffd700 1 -- 172.21.15.179:0/2279863683 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7fa71c004390 con 0x7fa7241027f0 2024-02-14T17:40:49.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.508+0000 7fa722ffd700 1 --2- 172.21.15.179:0/2279863683 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa70c038f80 0x7fa70c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:49.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.508+0000 7fa722ffd700 1 -- 172.21.15.179:0/2279863683 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa71c04cee0 con 0x7fa7241027f0 2024-02-14T17:40:49.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.508+0000 7fa7294b8700 1 --2- 172.21.15.179:0/2279863683 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa70c038f80 0x7fa70c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:49.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.509+0000 7fa7294b8700 1 --2- 172.21.15.179:0/2279863683 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa70c038f80 0x7fa70c03b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa718006fd0 tx=0x7fa718006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:49.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.514+0000 7fa722ffd700 1 -- 172.21.15.179:0/2279863683 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa71c00bba0 con 0x7fa7241027f0 2024-02-14T17:40:49.718 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.717+0000 7fa72bf1d700 1 -- 172.21.15.179:0/2279863683 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi179:172.21.15.179=b", "target": ["mon-mgr", ""]}) v1 -- 0x7fa72410afd0 con 0x7fa70c038f80 2024-02-14T17:40:49.725 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.723+0000 7fa722ffd700 1 -- 172.21.15.179:0/2279863683 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa72410afd0 con 0x7fa70c038f80 2024-02-14T17:40:49.725 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mon update... 2024-02-14T17:40:49.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.725+0000 7fa72bf1d700 1 -- 172.21.15.179:0/2279863683 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa70c038f80 msgr2=0x7fa70c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:49.727 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.725+0000 7fa72bf1d700 1 --2- 172.21.15.179:0/2279863683 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa70c038f80 0x7fa70c03b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa718006fd0 tx=0x7fa718006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:49.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.725+0000 7fa72bf1d700 1 -- 172.21.15.179:0/2279863683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7241027f0 msgr2=0x7fa7241965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:49.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.725+0000 7fa72bf1d700 1 --2- 172.21.15.179:0/2279863683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7241027f0 0x7fa7241965e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa71c006010 tx=0x7fa71c005710 comp rx=0 tx=0).stop 2024-02-14T17:40:49.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.726+0000 7fa72bf1d700 1 -- 172.21.15.179:0/2279863683 shutdown_connections 2024-02-14T17:40:49.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.726+0000 7fa72bf1d700 1 --2- 172.21.15.179:0/2279863683 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa70c038f80 0x7fa70c03b440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:49.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.726+0000 7fa72bf1d700 1 --2- 172.21.15.179:0/2279863683 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7241027f0 0x7fa7241965e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:49.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.726+0000 7fa72bf1d700 1 -- 172.21.15.179:0/2279863683 >> 172.21.15.179:0/2279863683 conn(0x7fa7240fc020 msgr2=0x7fa724105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:49.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.726+0000 7fa72bf1d700 1 -- 172.21.15.179:0/2279863683 shutdown_connections 2024-02-14T17:40:49.728 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:49.726+0000 7fa72bf1d700 1 -- 172.21.15.179:0/2279863683 wait complete. 2024-02-14T17:40:50.380 DEBUG:teuthology.orchestra.run.smithi100:mon.c> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.c.service 2024-02-14T17:40:50.382 DEBUG:teuthology.orchestra.run.smithi179:mon.b> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.b.service 2024-02-14T17:40:50.385 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T17:40:50.385 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T17:40:51.125 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:40:51.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi179:172.21.15.179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:51.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: Saving service mon spec with placement smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi179:172.21.15.179=b;count:3 2024-02-14T17:40:51.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:51.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:51.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:51.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:51.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:51.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:40:51.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:51.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: Deploying daemon mon.b on smithi179 2024-02-14T17:40:51.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:50 smithi100 ceph-mon[30138]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:53.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:53 smithi100 ceph-mon[30138]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.294+0000 7fac199c5700 1 -- 172.21.15.179:0/2558348252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac140ffce0 msgr2=0x7fac141000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.294+0000 7fac199c5700 1 --2- 172.21.15.179:0/2558348252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac140ffce0 0x7fac141000c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00b0a0 tx=0x7fac0c00b3b0 comp rx=0 tx=0).stop 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.294+0000 7fac199c5700 1 -- 172.21.15.179:0/2558348252 shutdown_connections 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.294+0000 7fac199c5700 1 --2- 172.21.15.179:0/2558348252 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac140ffce0 0x7fac141000c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.294+0000 7fac199c5700 1 -- 172.21.15.179:0/2558348252 >> 172.21.15.179:0/2558348252 conn(0x7fac140fb7e0 msgr2=0x7fac140fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.294+0000 7fac199c5700 1 -- 172.21.15.179:0/2558348252 shutdown_connections 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.294+0000 7fac199c5700 1 -- 172.21.15.179:0/2558348252 wait complete. 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.294+0000 7fac199c5700 1 Processor -- start 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.295+0000 7fac199c5700 1 -- start start 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.295+0000 7fac199c5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac140ffce0 0x7fac14100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:54.298 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.295+0000 7fac199c5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac14102b50 con 0x7fac140ffce0 2024-02-14T17:40:54.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.295+0000 7fac12ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac140ffce0 0x7fac14100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:54.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.295+0000 7fac12ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac140ffce0 0x7fac14100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.179:33322/0 (socket says 172.21.15.179:33322) 2024-02-14T17:40:54.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.295+0000 7fac12ffd700 1 -- 172.21.15.179:0/1467819771 learned_addr learned my addr 172.21.15.179:0/1467819771 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:40:54.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.295+0000 7fac12ffd700 1 -- 172.21.15.179:0/1467819771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c009d00 con 0x7fac140ffce0 2024-02-14T17:40:54.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.296+0000 7fac12ffd700 1 --2- 172.21.15.179:0/1467819771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac140ffce0 0x7fac14100c60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0062a0 tx=0x7fac0c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:54.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.296+0000 7fac189c3700 1 -- 172.21.15.179:0/1467819771 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac0c00e040 con 0x7fac140ffce0 2024-02-14T17:40:54.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.296+0000 7fac189c3700 1 -- 172.21.15.179:0/1467819771 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fac0c003c10 con 0x7fac140ffce0 2024-02-14T17:40:54.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.296+0000 7fac199c5700 1 -- 172.21.15.179:0/1467819771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac141011a0 con 0x7fac140ffce0 2024-02-14T17:40:54.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.296+0000 7fac189c3700 1 -- 172.21.15.179:0/1467819771 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fac0c01d470 con 0x7fac140ffce0 2024-02-14T17:40:54.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.296+0000 7fac199c5700 1 -- 172.21.15.179:0/1467819771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac141015c0 con 0x7fac140ffce0 2024-02-14T17:40:54.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.297+0000 7fac189c3700 1 -- 172.21.15.179:0/1467819771 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7fac0c019070 con 0x7fac140ffce0 2024-02-14T17:40:54.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.297+0000 7fac199c5700 1 -- 172.21.15.179:0/1467819771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac14105910 con 0x7fac140ffce0 2024-02-14T17:40:54.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.297+0000 7fac189c3700 1 --2- 172.21.15.179:0/1467819771 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fabfc039030 0x7fabfc03b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:40:54.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.297+0000 7fac189c3700 1 -- 172.21.15.179:0/1467819771 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fac0c04d180 con 0x7fac140ffce0 2024-02-14T17:40:54.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.297+0000 7fac127fc700 1 --2- 172.21.15.179:0/1467819771 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fabfc039030 0x7fabfc03b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:40:54.300 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.298+0000 7fac127fc700 1 --2- 172.21.15.179:0/1467819771 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fabfc039030 0x7fabfc03b4f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac08006fd0 tx=0x7fac08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:40:54.304 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.302+0000 7fac189c3700 1 -- 172.21.15.179:0/1467819771 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac0c026990 con 0x7fac140ffce0 2024-02-14T17:40:54.589 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.587+0000 7fac199c5700 1 -- 172.21.15.179:0/1467819771 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fac14102df0 con 0x7fac140ffce0 2024-02-14T17:40:54.590 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.588+0000 7fac189c3700 1 -- 172.21.15.179:0/1467819771 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+749 (secure 0 0 0) 0x7fac0c017020 con 0x7fac140ffce0 2024-02-14T17:40:54.590 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:40:54.590 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"d69a9998-cb5f-11ee-95b9-87774f69a715","modified":"2024-02-14T17:39:20.719111Z","created":"2024-02-14T17:39:20.719111Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-14T17:40:54.591 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.590+0000 7fac199c5700 1 -- 172.21.15.179:0/1467819771 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fabfc039030 msgr2=0x7fabfc03b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.590+0000 7fac199c5700 1 --2- 172.21.15.179:0/1467819771 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fabfc039030 0x7fabfc03b4f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac08006fd0 tx=0x7fac08006e40 comp rx=0 tx=0).stop 2024-02-14T17:40:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.590+0000 7fac199c5700 1 -- 172.21.15.179:0/1467819771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac140ffce0 msgr2=0x7fac14100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:40:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.590+0000 7fac199c5700 1 --2- 172.21.15.179:0/1467819771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac140ffce0 0x7fac14100c60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac0c0062a0 tx=0x7fac0c00bd40 comp rx=0 tx=0).stop 2024-02-14T17:40:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.590+0000 7fac199c5700 1 -- 172.21.15.179:0/1467819771 shutdown_connections 2024-02-14T17:40:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.590+0000 7fac199c5700 1 --2- 172.21.15.179:0/1467819771 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fabfc039030 0x7fabfc03b4f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.590+0000 7fac199c5700 1 --2- 172.21.15.179:0/1467819771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac140ffce0 0x7fac14100c60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:40:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.590+0000 7fac199c5700 1 -- 172.21.15.179:0/1467819771 >> 172.21.15.179:0/1467819771 conn(0x7fac140fb7e0 msgr2=0x7fac14109530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:40:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.590+0000 7fac199c5700 1 -- 172.21.15.179:0/1467819771 shutdown_connections 2024-02-14T17:40:54.592 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:40:54.590+0000 7fac199c5700 1 -- 172.21.15.179:0/1467819771 wait complete. 2024-02-14T17:40:54.593 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-02-14T17:40:54.634 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 podman[33562]: 2024-02-14 17:40:54.627149714 +0000 UTC m=+0.344263742 container init c6063f9d54e1c0525551323259469b5cdf481ec5c1634c633c8e165b5a334ef0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-14T17:40:54.635 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 podman[33562]: 2024-02-14 17:40:54.633319676 +0000 UTC m=+0.350433704 container start c6063f9d54e1c0525551323259469b5cdf481ec5c1634c633c8e165b5a334ef0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-b, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-14T17:40:55.057 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T17:40:55.057 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev), process ceph-mon, pid 7 2024-02-14T17:40:55.057 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: pidfile_write: ignore empty --pid-file 2024-02-14T17:40:55.057 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: load: jerasure load: lrc 2024-02-14T17:40:55.057 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: RocksDB version: 7.9.2 2024-02-14T17:40:55.057 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Git sha 0 2024-02-14T17:40:55.057 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Compile date 2024-02-14 11:59:27 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: DB SUMMARY 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: DB Session ID: V1QCIEP8H6XSTF0UJUY8 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: CURRENT file: CURRENT 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.error_if_exists: 0 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.create_if_missing: 0 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.paranoid_checks: 1 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T17:40:55.058 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.env: 0x55666d743ae0 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.info_log: 0x55666f4898c0 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.statistics: (nil) 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.use_fsync: 0 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_log_file_size: 0 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.allow_fallocate: 1 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T17:40:55.059 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.use_direct_reads: 0 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.db_log_dir: 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.wal_dir: 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.write_buffer_manager: 0x55666f45d400 2024-02-14T17:40:55.060 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.unordered_write: 0 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.row_cache: None 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.wal_filter: None 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T17:40:55.061 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.two_write_queues: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.wal_compression: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.atomic_flush: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.log_readahead_size: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T17:40:55.062 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T17:40:55.064 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_background_jobs: 2 2024-02-14T17:40:55.064 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_background_compactions: -1 2024-02-14T17:40:55.064 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_subcompactions: 1 2024-02-14T17:40:55.064 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_open_files: -1 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_background_flushes: -1 2024-02-14T17:40:55.065 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Compression algorithms supported: 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: kZSTD supported: 0 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: kXpressCompression supported: 0 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: kZlibCompression supported: 1 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: kSnappyCompression supported: 1 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: kLZ4Compression supported: 1 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: kBZip2Compression supported: 0 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T17:40:55.066 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.merge_operator: 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_filter: None 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55666f4899c0) 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout: cache_index_and_filter_blocks: 1 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout: pin_top_level_index_and_filter: 1 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout: index_type: 0 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout: data_block_index_type: 0 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout: index_shortening: 1 2024-02-14T17:40:55.067 INFO:journalctl@ceph.mon.b.smithi179.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: checksum: 4 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: no_block_cache: 0 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: block_cache: 0x55666f48d090 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: block_cache_name: BinnedLRUCache 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: block_cache_options: 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: capacity : 536870912 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: num_shard_bits : 4 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: strict_capacity_limit : 0 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: high_pri_pool_ratio: 0.000 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: block_cache_compressed: (nil) 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: persistent_cache: (nil) 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: block_size: 4096 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: block_size_deviation: 10 2024-02-14T17:40:55.068 INFO:journalctl@ceph.mon.b.smithi179.stdout: block_restart_interval: 16 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: index_block_restart_interval: 1 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: metadata_block_size: 4096 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: partition_filters: 0 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: use_delta_encoding: 1 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: filter_policy: bloomfilter 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: whole_key_filtering: 1 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: verify_compression: 0 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: read_amp_bytes_per_bit: 0 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: format_version: 5 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: enable_index_compression: 1 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: block_align: 0 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: max_auto_readahead_size: 262144 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: prepopulate_block_cache: 0 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: initial_auto_readahead_size: 8192 2024-02-14T17:40:55.069 INFO:journalctl@ceph.mon.b.smithi179.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compression: NoCompression 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.num_levels: 7 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T17:40:55.070 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:40:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T17:40:55.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T17:40:55.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.table_properties_collectors: 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.inplace_update_support: 0 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.bloom_locality: 0 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.max_successive_merges: 0 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T17:40:55.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T17:40:55.075 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T17:40:55.075 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.ttl: 2592000 2024-02-14T17:40:55.075 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T17:40:55.075 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T17:40:55.075 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T17:40:55.075 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.enable_blob_files: false 2024-02-14T17:40:55.075 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.min_blob_size: 0 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a8c955c9-1f35-4f0a-8ce3-a28124ccd242 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932454662814, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-14T17:40:55.076 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932454663238, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707932454, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a8c955c9-1f35-4f0a-8ce3-a28124ccd242", "db_session_id": "V1QCIEP8H6XSTF0UJUY8", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932454663315, "job": 1, "event": "recovery_finished"} 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55666f57e000 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: DB pointer 0x55666f568000 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout: ** DB Stats ** 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:40:55.077 INFO:journalctl@ceph.mon.b.smithi179.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: ** Compaction Stats [default] ** 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: ** Compaction Stats [default] ** 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:40:55.078 INFO:journalctl@ceph.mon.b.smithi179.stdout: 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: Block cache BinnedLRUCache@0x55666f48d090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: 2024-02-14T17:40:55.079 INFO:journalctl@ceph.mon.b.smithi179.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T17:40:55.080 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-14T17:40:55.080 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: using public_addr v2:172.21.15.179:0/0 -> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-02-14T17:40:55.080 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: starting mon.b rank -1 at public addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] at bind addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:40:55.080 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(???) e0 preinit fsid d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:40:55.080 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).mds e1 new map 2024-02-14T17:40:55.080 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-14T17:40:55.080 INFO:journalctl@ceph.mon.b.smithi179.stdout: e1 2024-02-14T17:40:55.081 INFO:journalctl@ceph.mon.b.smithi179.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T17:40:55.081 INFO:journalctl@ceph.mon.b.smithi179.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T17:40:55.081 INFO:journalctl@ceph.mon.b.smithi179.stdout: legacy client fscid: -1 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout: 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout: No filesystems configured 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:55.082 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209095686' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: Updating smithi179:/etc/ceph/ceph.conf 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:40:55.083 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209095686' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi179:172.21.15.179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: Saving service mon spec with placement smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi179:172.21.15.179=b;count:3 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:40:55.084 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:55.085 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:40:55.085 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:40:55.085 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:40:55.085 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:40:55.085 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: Deploying daemon mon.b on smithi179 2024-02-14T17:40:55.085 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:55.085 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:40:55.085 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 ceph-mon[33633]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-14T17:40:55.085 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 bash[33562]: c6063f9d54e1c0525551323259469b5cdf481ec5c1634c633c8e165b5a334ef0 2024-02-14T17:40:55.086 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:54 smithi179 systemd[1]: Started Ceph mon.b for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:40:56.411 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T17:40:56.411 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T17:40:56.657 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:41:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:41:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: mon.a calling monitor election 2024-02-14T17:41:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: mon.b calling monitor election 2024-02-14T17:41:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: fsmap 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: mgrmap e13: y(active, since 41s) 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: overall HEALTH_OK 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:00.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:40:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:41:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: mon.a calling monitor election 2024-02-14T17:41:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: mon.b calling monitor election 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: fsmap 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: mgrmap e13: y(active, since 41s) 2024-02-14T17:41:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: overall HEALTH_OK 2024-02-14T17:41:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:40:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:00 smithi100 ceph-mon[30138]: Deploying daemon mon.c on smithi100 2024-02-14T17:41:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:00 smithi100 ceph-mon[30138]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:00.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:00 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:00 smithi179 ceph-mon[33633]: Deploying daemon mon.c on smithi100 2024-02-14T17:41:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:00 smithi179 ceph-mon[33633]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:01.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:00 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:01.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.251+0000 7fa1e2908700 1 -- 172.21.15.179:0/4156765686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1c40035c0 msgr2=0x7fa1c4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:01.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.251+0000 7fa1e2908700 1 --2- 172.21.15.179:0/4156765686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1c40035c0 0x7fa1c4005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc1005a0 tx=0x7fa1cc00feb0 comp rx=0 tx=0).stop 2024-02-14T17:41:01.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.251+0000 7fa1e2908700 1 -- 172.21.15.179:0/4156765686 shutdown_connections 2024-02-14T17:41:01.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.251+0000 7fa1e2908700 1 --2- 172.21.15.179:0/4156765686 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1c40035c0 0x7fa1c4005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:01.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.251+0000 7fa1e2908700 1 -- 172.21.15.179:0/4156765686 >> 172.21.15.179:0/4156765686 conn(0x7fa1dc0fc180 msgr2=0x7fa1dc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:01.253 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.251+0000 7fa1e2908700 1 -- 172.21.15.179:0/4156765686 shutdown_connections 2024-02-14T17:41:01.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.251+0000 7fa1e2908700 1 -- 172.21.15.179:0/4156765686 wait complete. 2024-02-14T17:41:01.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.252+0000 7fa1e2908700 1 Processor -- start 2024-02-14T17:41:01.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.252+0000 7fa1e2908700 1 -- start start 2024-02-14T17:41:01.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1e2908700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1c40035c0 0x7fa1dc197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:01.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1e2908700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1dc197ad0 0x7fa1dc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:01.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1e2908700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc06adb0 con 0x7fa1c40035c0 2024-02-14T17:41:01.254 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1e2908700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc1a0600 con 0x7fa1dc197ad0 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1db7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1dc197ad0 0x7fa1dc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1db7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1dc197ad0 0x7fa1dc197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:39850/0 (socket says 172.21.15.179:39850) 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1db7fe700 1 -- 172.21.15.179:0/841866115 learned_addr learned my addr 172.21.15.179:0/841866115 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1dbfff700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1c40035c0 0x7fa1dc197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1db7fe700 1 -- 172.21.15.179:0/841866115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1dc197ad0 msgr2=0x7fa1dc197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1db7fe700 1 -- 172.21.15.179:0/841866115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1dc197ad0 msgr2=0x7fa1dc197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1db7fe700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1dc197ad0 0x7fa1dc197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1db7fe700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1dc197ad0 0x7fa1dc197f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1dbfff700 1 -- 172.21.15.179:0/841866115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1dc197ad0 msgr2=0x7fa1dc197f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1dbfff700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1dc197ad0 0x7fa1dc197f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:01.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1dbfff700 1 -- 172.21.15.179:0/841866115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc00f900 con 0x7fa1c40035c0 2024-02-14T17:41:01.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.253+0000 7fa1dbfff700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1c40035c0 0x7fa1dc197590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00f800 tx=0x7fa1cc004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:01.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.254+0000 7fa1d97fa700 1 -- 172.21.15.179:0/841866115 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa1cc013b00 con 0x7fa1c40035c0 2024-02-14T17:41:01.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.254+0000 7fa1e2908700 1 -- 172.21.15.179:0/841866115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc1a0800 con 0x7fa1c40035c0 2024-02-14T17:41:01.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.254+0000 7fa1d97fa700 1 -- 172.21.15.179:0/841866115 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa1cc0044b0 con 0x7fa1c40035c0 2024-02-14T17:41:01.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.254+0000 7fa1d97fa700 1 -- 172.21.15.179:0/841866115 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fa1cc003bb0 con 0x7fa1c40035c0 2024-02-14T17:41:01.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.254+0000 7fa1e2908700 1 -- 172.21.15.179:0/841866115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc1a0d50 con 0x7fa1c40035c0 2024-02-14T17:41:01.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.255+0000 7fa1d97fa700 1 -- 172.21.15.179:0/841866115 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7fa1cc003d50 con 0x7fa1c40035c0 2024-02-14T17:41:01.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.255+0000 7fa1e2908700 1 -- 172.21.15.179:0/841866115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc18f220 con 0x7fa1c40035c0 2024-02-14T17:41:01.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.255+0000 7fa1d97fa700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1c8038d20 0x7fa1c803b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:01.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.255+0000 7fa1d97fa700 1 -- 172.21.15.179:0/841866115 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa1cc04e660 con 0x7fa1c40035c0 2024-02-14T17:41:01.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.255+0000 7fa1db7fe700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1c8038d20 0x7fa1c803b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:01.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.256+0000 7fa1db7fe700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1c8038d20 0x7fa1c803b1e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009180 tx=0x7fa1d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:01.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.261+0000 7fa1d97fa700 1 -- 172.21.15.179:0/841866115 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa1cc0107a0 con 0x7fa1c40035c0 2024-02-14T17:41:01.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.535+0000 7fa1e2908700 1 -- 172.21.15.179:0/841866115 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa1dc06a420 con 0x7fa1c40035c0 2024-02-14T17:41:01.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.535+0000 7fa1d97fa700 1 -- 172.21.15.179:0/841866115 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7fa1cc01a390 con 0x7fa1c40035c0 2024-02-14T17:41:01.537 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:41:01.537 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"d69a9998-cb5f-11ee-95b9-87774f69a715","modified":"2024-02-14T17:40:54.684532Z","created":"2024-02-14T17:39:20.719111Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 -- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1c8038d20 msgr2=0x7fa1c803b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1c8038d20 0x7fa1c803b1e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009180 tx=0x7fa1d0008040 comp rx=0 tx=0).stop 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 -- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1c40035c0 msgr2=0x7fa1dc197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1c40035c0 0x7fa1dc197590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00f800 tx=0x7fa1cc004f40 comp rx=0 tx=0).stop 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 -- 172.21.15.179:0/841866115 shutdown_connections 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa1c40035c0 0x7fa1dc197590 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fa1c8038d20 0x7fa1c803b1e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 --2- 172.21.15.179:0/841866115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1dc197ad0 0x7fa1dc197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 -- 172.21.15.179:0/841866115 >> 172.21.15.179:0/841866115 conn(0x7fa1dc0fc180 msgr2=0x7fa1dc0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 -- 172.21.15.179:0/841866115 shutdown_connections 2024-02-14T17:41:01.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:01.538+0000 7fa1e2908700 1 -- 172.21.15.179:0/841866115 wait complete. 2024-02-14T17:41:01.541 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2024-02-14T17:41:02.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/841866115' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:41:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/841866115' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:41:03.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:02 smithi179 ceph-mon[33633]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.122 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-14T17:41:03.122 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-14T17:41:03.137 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:02 smithi100 ceph-mon[30138]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.138 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:02 smithi100 systemd[1]: Starting Ceph mon.c for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:41:03.364 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:41:03.398 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 podman[39519]: 2024-02-14 17:41:03.136357419 +0000 UTC m=+0.114086208 container create e5501fd1d64a369bf51fb0e4b5b322da7bf7cdba97439067fba6f43628c293f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-14T17:41:03.398 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 podman[39519]: 2024-02-14 17:41:03.040699518 +0000 UTC m=+0.018428325 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 podman[39519]: 2024-02-14 17:41:03.347346874 +0000 UTC m=+0.325075664 container init e5501fd1d64a369bf51fb0e4b5b322da7bf7cdba97439067fba6f43628c293f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-c, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 podman[39519]: 2024-02-14 17:41:03.352638222 +0000 UTC m=+0.330367015 container start e5501fd1d64a369bf51fb0e4b5b322da7bf7cdba97439067fba6f43628c293f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: set uid:gid to 167:167 (ceph:ceph) 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: ceph version 19.0.0-1430-g8727ce93 (8727ce939fc044a98f215e29696bb1433f5758d1) squid (dev), process ceph-mon, pid 7 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pidfile_write: ignore empty --pid-file 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: load: jerasure load: lrc 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: RocksDB version: 7.9.2 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Git sha 0 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Compile date 2024-02-14 11:59:27 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: DB SUMMARY 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: DB Session ID: IWMEG6TIA7I4SRK4S9K7 2024-02-14T17:41:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: CURRENT file: CURRENT 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: IDENTITY file: IDENTITY 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.error_if_exists: 0 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.create_if_missing: 0 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.paranoid_checks: 1 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.env: 0x55c40a1c2ae0 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.fs: PosixFileSystem 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.info_log: 0x55c40c6538c0 2024-02-14T17:41:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_file_opening_threads: 16 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.statistics: (nil) 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.use_fsync: 0 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_log_file_size: 0 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.keep_log_file_num: 1000 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.recycle_log_file_num: 0 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.allow_fallocate: 1 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.allow_mmap_reads: 0 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.allow_mmap_writes: 0 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.use_direct_reads: 0 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.create_missing_column_families: 0 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.db_log_dir: 2024-02-14T17:41:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.wal_dir: 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.advise_random_on_open: 1 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.db_write_buffer_size: 0 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.write_buffer_manager: 0x55c40c627400 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-14T17:41:03.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.rate_limiter: (nil) 2024-02-14T17:41:03.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-14T17:41:03.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.wal_recovery_mode: 2 2024-02-14T17:41:03.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.enable_thread_tracking: 0 2024-02-14T17:41:03.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.enable_pipelined_write: 0 2024-02-14T17:41:03.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.unordered_write: 0 2024-02-14T17:41:03.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-14T17:41:03.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-14T17:41:03.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.row_cache: None 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.wal_filter: None 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.allow_ingest_behind: 0 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.two_write_queues: 0 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.manual_wal_flush: 0 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.wal_compression: 0 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.atomic_flush: 0 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.log_readahead_size: 0 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-14T17:41:03.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.best_efforts_recovery: 0 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.allow_data_in_errors: 0 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.db_host_id: __hostname__ 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_background_jobs: 2 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_background_compactions: -1 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_subcompactions: 1 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-14T17:41:03.406 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_total_wal_size: 0 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_open_files: -1 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bytes_per_sync: 0 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_readahead_size: 0 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_background_flushes: -1 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Compression algorithms supported: 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: kZSTD supported: 0 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: kXpressCompression supported: 0 2024-02-14T17:41:03.407 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: kLZ4HCCompression supported: 1 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: kZlibCompression supported: 1 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: kSnappyCompression supported: 1 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: kLZ4Compression supported: 1 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: kBZip2Compression supported: 0 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.merge_operator: 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_filter: None 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_filter_factory: None 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.sst_partitioner_factory: None 2024-02-14T17:41:03.408 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c40c6539c0) 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: cache_index_and_filter_blocks: 1 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: pin_top_level_index_and_filter: 1 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: index_type: 0 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: data_block_index_type: 0 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: index_shortening: 1 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: checksum: 4 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: no_block_cache: 0 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: block_cache: 0x55c40c657090 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: block_cache_name: BinnedLRUCache 2024-02-14T17:41:03.409 INFO:journalctl@ceph.mon.c.smithi100.stdout: block_cache_options: 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: capacity : 536870912 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: num_shard_bits : 4 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: strict_capacity_limit : 0 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: high_pri_pool_ratio: 0.000 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: block_cache_compressed: (nil) 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: persistent_cache: (nil) 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: block_size: 4096 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: block_size_deviation: 10 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: block_restart_interval: 16 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: index_block_restart_interval: 1 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: metadata_block_size: 4096 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: partition_filters: 0 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: use_delta_encoding: 1 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: filter_policy: bloomfilter 2024-02-14T17:41:03.410 INFO:journalctl@ceph.mon.c.smithi100.stdout: whole_key_filtering: 1 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout: verify_compression: 0 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout: read_amp_bytes_per_bit: 0 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout: format_version: 5 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout: enable_index_compression: 1 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout: block_align: 0 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout: max_auto_readahead_size: 262144 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout: prepopulate_block_cache: 0 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout: initial_auto_readahead_size: 8192 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout: num_file_reads_for_auto_readahead: 2 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.write_buffer_size: 33554432 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_write_buffer_number: 2 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compression: NoCompression 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bottommost_compression: Disabled 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.prefix_extractor: nullptr 2024-02-14T17:41:03.411 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.num_levels: 7 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:41:03.412 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compression_opts.level: 32767 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compression_opts.strategy: 0 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compression_opts.enabled: false 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.target_file_size_base: 67108864 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-14T17:41:03.413 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.arena_block_size: 1048576 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-14T17:41:03.414 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.disable_auto_compactions: 0 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.table_properties_collectors: 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.inplace_update_support: 0 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-14T17:41:03.415 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.bloom_locality: 0 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.max_successive_merges: 0 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.paranoid_file_checks: 0 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.force_consistency_checks: 1 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.report_bg_io_stats: 0 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.ttl: 2592000 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-14T17:41:03.416 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.enable_blob_files: false 2024-02-14T17:41:03.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.min_blob_size: 0 2024-02-14T17:41:03.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.blob_file_size: 268435456 2024-02-14T17:41:03.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-14T17:41:03.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-14T17:41:03.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-14T17:41:03.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-14T17:41:03.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-14T17:41:03.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.blob_file_starting_level: 0 2024-02-14T17:41:03.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-14T17:41:03.417 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-14T17:41:03.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-14T17:41:03.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b839b355-7302-450e-a1a8-00fbf3c37561 2024-02-14T17:41:03.418 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932463382526, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-14T17:41:03.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-14T17:41:03.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932463382917, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707932463, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b839b355-7302-450e-a1a8-00fbf3c37561", "db_session_id": "IWMEG6TIA7I4SRK4S9K7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-14T17:41:03.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707932463382997, "job": 1, "event": "recovery_finished"} 2024-02-14T17:41:03.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-14T17:41:03.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-14T17:41:03.419 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c40c748000 2024-02-14T17:41:03.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: DB pointer 0x55c40c732000 2024-02-14T17:41:03.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-14T17:41:03.420 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-14T17:41:03.420 INFO:journalctl@ceph.mon.c.smithi100.stdout: ** DB Stats ** 2024-02-14T17:41:03.420 INFO:journalctl@ceph.mon.c.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:41:03.421 INFO:journalctl@ceph.mon.c.smithi100.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-14T17:41:03.421 INFO:journalctl@ceph.mon.c.smithi100.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:41:03.421 INFO:journalctl@ceph.mon.c.smithi100.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:41:03.421 INFO:journalctl@ceph.mon.c.smithi100.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-14T17:41:03.421 INFO:journalctl@ceph.mon.c.smithi100.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-14T17:41:03.421 INFO:journalctl@ceph.mon.c.smithi100.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-14T17:41:03.422 INFO:journalctl@ceph.mon.c.smithi100.stdout: 2024-02-14T17:41:03.422 INFO:journalctl@ceph.mon.c.smithi100.stdout: ** Compaction Stats [default] ** 2024-02-14T17:41:03.422 INFO:journalctl@ceph.mon.c.smithi100.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:41:03.422 INFO:journalctl@ceph.mon.c.smithi100.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-14T17:41:03.422 INFO:journalctl@ceph.mon.c.smithi100.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:41:03.422 INFO:journalctl@ceph.mon.c.smithi100.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:41:03.423 INFO:journalctl@ceph.mon.c.smithi100.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:41:03.423 INFO:journalctl@ceph.mon.c.smithi100.stdout: 2024-02-14T17:41:03.423 INFO:journalctl@ceph.mon.c.smithi100.stdout: ** Compaction Stats [default] ** 2024-02-14T17:41:03.423 INFO:journalctl@ceph.mon.c.smithi100.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-14T17:41:03.423 INFO:journalctl@ceph.mon.c.smithi100.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-14T17:41:03.423 INFO:journalctl@ceph.mon.c.smithi100.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-14T17:41:03.424 INFO:journalctl@ceph.mon.c.smithi100.stdout: 2024-02-14T17:41:03.424 INFO:journalctl@ceph.mon.c.smithi100.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-14T17:41:03.424 INFO:journalctl@ceph.mon.c.smithi100.stdout: 2024-02-14T17:41:03.424 INFO:journalctl@ceph.mon.c.smithi100.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-14T17:41:03.424 INFO:journalctl@ceph.mon.c.smithi100.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-14T17:41:03.425 INFO:journalctl@ceph.mon.c.smithi100.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-14T17:41:03.425 INFO:journalctl@ceph.mon.c.smithi100.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-14T17:41:03.425 INFO:journalctl@ceph.mon.c.smithi100.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-14T17:41:03.425 INFO:journalctl@ceph.mon.c.smithi100.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-14T17:41:03.426 INFO:journalctl@ceph.mon.c.smithi100.stdout: Cumulative compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:41:03.426 INFO:journalctl@ceph.mon.c.smithi100.stdout: Interval compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-14T17:41:03.426 INFO:journalctl@ceph.mon.c.smithi100.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-14T17:41:03.426 INFO:journalctl@ceph.mon.c.smithi100.stdout: Block cache BinnedLRUCache@0x55c40c657090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-14T17:41:03.426 INFO:journalctl@ceph.mon.c.smithi100.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-14T17:41:03.426 INFO:journalctl@ceph.mon.c.smithi100.stdout: 2024-02-14T17:41:03.427 INFO:journalctl@ceph.mon.c.smithi100.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-14T17:41:03.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-14T17:41:03.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: using public_addrv [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] 2024-02-14T17:41:03.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: starting mon.c rank -1 at public addrs [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] at bind addrs [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:41:03.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(???) e0 preinit fsid d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:41:03.427 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).mds e1 new map 2024-02-14T17:41:03.428 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-14T17:41:03.428 INFO:journalctl@ceph.mon.c.smithi100.stdout: e1 2024-02-14T17:41:03.428 INFO:journalctl@ceph.mon.c.smithi100.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-14T17:41:03.428 INFO:journalctl@ceph.mon.c.smithi100.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-14T17:41:03.428 INFO:journalctl@ceph.mon.c.smithi100.stdout: legacy client fscid: -1 2024-02-14T17:41:03.428 INFO:journalctl@ceph.mon.c.smithi100.stdout: 2024-02-14T17:41:03.429 INFO:journalctl@ceph.mon.c.smithi100.stdout: No filesystems configured 2024-02-14T17:41:03.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-14T17:41:03.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-14T17:41:03.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-14T17:41:03.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-14T17:41:03.429 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-14T17:41:03.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-14T17:41:03.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-14T17:41:03.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:41:03.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:41:03.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-14T17:41:03.430 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:41:03.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.431 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209095686' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-14T17:41:03.432 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.432 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.432 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.432 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.432 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:03.432 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:03.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:03.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: Updating smithi179:/etc/ceph/ceph.conf 2024-02-14T17:41:03.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:41:03.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209095686' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-14T17:41:03.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:41:03.433 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:41:03.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:41:03.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi179:172.21.15.179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:03.434 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: Saving service mon spec with placement smithi100:172.21.15.100=a;smithi100:[v2:172.21.15.100:3301,v1:172.21.15.100:6790]=c;smithi179:172.21.15.179=b;count:3 2024-02-14T17:41:03.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:03.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:03.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:03.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.435 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:03.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:03.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: Deploying daemon mon.b on smithi179 2024-02-14T17:41:03.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:41:03.436 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.a calling monitor election 2024-02-14T17:41:03.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:03.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:03.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:03.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.b calling monitor election 2024-02-14T17:41:03.437 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-14T17:41:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: monmap e2: 2 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:41:03.438 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: fsmap 2024-02-14T17:41:03.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:41:03.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mgrmap e13: y(active, since 41s) 2024-02-14T17:41:03.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: overall HEALTH_OK 2024-02-14T17:41:03.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.439 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: Deploying daemon mon.c on smithi100 2024-02-14T17:41:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/841866115' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:41:03.440 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:03.441 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 ceph-mon[39557]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-14T17:41:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 bash[39519]: e5501fd1d64a369bf51fb0e4b5b322da7bf7cdba97439067fba6f43628c293f3 2024-02-14T17:41:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:03 smithi100 systemd[1]: Started Ceph mon.c for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:41:08.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.453+0000 7f46115fe700 1 -- 172.21.15.179:0/3678874444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45f40035c0 msgr2=0x7f45f4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:08.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.453+0000 7f46115fe700 1 --2- 172.21.15.179:0/3678874444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45f40035c0 0x7f45f4005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f460c100d70 tx=0x7f45fc014ec0 comp rx=0 tx=0).stop 2024-02-14T17:41:08.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.453+0000 7f46115fe700 1 -- 172.21.15.179:0/3678874444 shutdown_connections 2024-02-14T17:41:08.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.453+0000 7f46115fe700 1 --2- 172.21.15.179:0/3678874444 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f45f40035c0 0x7f45f4005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:08.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.453+0000 7f46115fe700 1 -- 172.21.15.179:0/3678874444 >> 172.21.15.179:0/3678874444 conn(0x7f460c076d60 msgr2=0x7f460c077170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:08.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.453+0000 7f46115fe700 1 -- 172.21.15.179:0/3678874444 shutdown_connections 2024-02-14T17:41:08.455 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.453+0000 7f46115fe700 1 -- 172.21.15.179:0/3678874444 wait complete. 2024-02-14T17:41:08.457 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.455+0000 7f46115fe700 1 Processor -- start 2024-02-14T17:41:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.455+0000 7f46115fe700 1 -- start start 2024-02-14T17:41:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.455+0000 7f46115fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45f40035c0 0x7f460c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.455+0000 7f46115fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f460c19c9a0 0x7f460c196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.455+0000 7f46115fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c196b60 0x7f460c197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.455+0000 7f46115fe700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c19d060 con 0x7f460c196b60 2024-02-14T17:41:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.455+0000 7f46115fe700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1a3330 con 0x7f460c19c9a0 2024-02-14T17:41:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.455+0000 7f46115fe700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1a3470 con 0x7f45f40035c0 2024-02-14T17:41:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.455+0000 7f460a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f460c19c9a0 0x7f460c196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460affd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45f40035c0 0x7f460c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:08.458 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460a7fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f460c19c9a0 0x7f460c196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44530/0 (socket says 172.21.15.179:44530) 2024-02-14T17:41:08.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460a7fc700 1 -- 172.21.15.179:0/1541481652 learned_addr learned my addr 172.21.15.179:0/1541481652 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:41:08.459 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460b7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c196b60 0x7f460c197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460affd700 1 -- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45f40035c0 msgr2=0x7f460c19c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460affd700 1 -- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45f40035c0 msgr2=0x7f460c19c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460affd700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45f40035c0 0x7f460c19c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460a7fc700 1 -- 172.21.15.179:0/1541481652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f460c19c9a0 msgr2=0x7f460c196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460a7fc700 1 -- 172.21.15.179:0/1541481652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f460c19c9a0 msgr2=0x7f460c196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460a7fc700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f460c19c9a0 0x7f460c196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460affd700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45f40035c0 0x7f460c19c460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460a7fc700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f460c19c9a0 0x7f460c196620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460b7fe700 1 -- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45f40035c0 msgr2=0x7f460c19c460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460b7fe700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45f40035c0 0x7f460c19c460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460b7fe700 1 -- 172.21.15.179:0/1541481652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f460c19c9a0 msgr2=0x7f460c196620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460b7fe700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f460c19c9a0 0x7f460c196620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:08.460 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460b7fe700 1 -- 172.21.15.179:0/1541481652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc014910 con 0x7f460c196b60 2024-02-14T17:41:08.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.456+0000 7f460b7fe700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c196b60 0x7f460c197010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f460400be00 tx=0x7f460400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:08.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.457+0000 7f45f3fff700 1 -- 172.21.15.179:0/1541481652 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400cc90 con 0x7f460c196b60 2024-02-14T17:41:08.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.457+0000 7f45f3fff700 1 -- 172.21.15.179:0/1541481652 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f460400cdf0 con 0x7f460c196b60 2024-02-14T17:41:08.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.457+0000 7f46115fe700 1 -- 172.21.15.179:0/1541481652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c1a3750 con 0x7f460c196b60 2024-02-14T17:41:08.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.457+0000 7f45f3fff700 1 -- 172.21.15.179:0/1541481652 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040125e0 con 0x7f460c196b60 2024-02-14T17:41:08.461 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.457+0000 7f46115fe700 1 -- 172.21.15.179:0/1541481652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c1a3ca0 con 0x7f460c196b60 2024-02-14T17:41:08.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.458+0000 7f45f3fff700 1 -- 172.21.15.179:0/1541481652 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f4604012800 con 0x7f460c196b60 2024-02-14T17:41:08.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.458+0000 7f45f3fff700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f45f8039410 0x7f45f803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:08.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.458+0000 7f45f3fff700 1 -- 172.21.15.179:0/1541481652 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f460404b540 con 0x7f460c196b60 2024-02-14T17:41:08.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.458+0000 7f46115fe700 1 -- 172.21.15.179:0/1541481652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c0632a0 con 0x7f460c196b60 2024-02-14T17:41:08.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.458+0000 7f460affd700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f45f8039410 0x7f45f803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:08.462 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.459+0000 7f460affd700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f45f8039410 0x7f45f803b8d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f45fc001eb0 tx=0x7f45fc00ff90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:08.465 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.464+0000 7f45f3fff700 1 -- 172.21.15.179:0/1541481652 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4604011890 con 0x7f460c196b60 2024-02-14T17:41:08.751 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.750+0000 7f46115fe700 1 -- 172.21.15.179:0/1541481652 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f460c108af0 con 0x7f460c196b60 2024-02-14T17:41:08.752 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:41:08.752 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: mon.a calling monitor election 2024-02-14T17:41:08.752 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: mon.b calling monitor election 2024-02-14T17:41:08.752 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:08.752 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.752 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:08.752 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.752 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.752 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: mon.c calling monitor election 2024-02-14T17:41:08.752 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: fsmap 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: mgrmap e13: y(active, since 50s) 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: overall HEALTH_OK 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.753 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.754 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.751+0000 7f45f3fff700 1 -- 172.21.15.179:0/1541481652 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1289 (secure 0 0 0) 0x7f46040112a0 con 0x7f460c196b60 2024-02-14T17:41:08.754 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-02-14T17:41:08.754 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":3,"fsid":"d69a9998-cb5f-11ee-95b9-87774f69a715","modified":"2024-02-14T17:41:03.402463Z","created":"2024-02-14T17:39:20.719111Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3301","nonce":0},{"type":"v1","addr":"172.21.15.100:6790","nonce":0}]},"addr":"172.21.15.100:6790/0","public_addr":"172.21.15.100:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 -- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f45f8039410 msgr2=0x7f45f803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f45f8039410 0x7f45f803b8d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f45fc001eb0 tx=0x7f45fc00ff90 comp rx=0 tx=0).stop 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 -- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c196b60 msgr2=0x7f460c197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c196b60 0x7f460c197010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f460400be00 tx=0x7f460400bee0 comp rx=0 tx=0).stop 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 -- 172.21.15.179:0/1541481652 shutdown_connections 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f460c196b60 0x7f460c197010 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f45f40035c0 0x7f460c19c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f45f8039410 0x7f45f803b8d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 --2- 172.21.15.179:0/1541481652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f460c19c9a0 0x7f460c196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 -- 172.21.15.179:0/1541481652 >> 172.21.15.179:0/1541481652 conn(0x7f460c076d60 msgr2=0x7f460c104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 -- 172.21.15.179:0/1541481652 shutdown_connections 2024-02-14T17:41:08.755 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:08.753+0000 7f46115fe700 1 -- 172.21.15.179:0/1541481652 wait complete. 2024-02-14T17:41:08.756 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 3 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: mon.a calling monitor election 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: mon.b calling monitor election 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: mon.c calling monitor election 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: fsmap 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: mgrmap e13: y(active, since 50s) 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: overall HEALTH_OK 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: mon.a calling monitor election 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: mon.b calling monitor election 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: mon.c calling monitor election 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: monmap e3: 3 mons at {a=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],b=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0],c=[v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-14T17:41:08.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: fsmap 2024-02-14T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: osdmap e4: 0 total, 0 up, 0 in 2024-02-14T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: mgrmap e13: y(active, since 50s) 2024-02-14T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: overall HEALTH_OK 2024-02-14T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:08.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:09.381 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-14T17:41:09.381 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph config generate-minimal-conf 2024-02-14T17:41:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:09 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/1541481652' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:41:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:09 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:09 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/1541481652' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:41:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:09 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:09.741 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:09 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:09.741 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/1541481652' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-14T17:41:09.742 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:09 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:41:09.983 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:41:11.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:11.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.261 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: Updating smithi179:/etc/ceph/ceph.conf 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:41:11.262 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: Updating smithi179:/etc/ceph/ceph.conf 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.263 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.264 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.264 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.264 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.264 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.264 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.264 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:11.264 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:41:11.264 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: Updating smithi179:/etc/ceph/ceph.conf 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:11.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:11.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:41:11.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:12.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:12 smithi100 ceph-mon[30138]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:41:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:12 smithi100 ceph-mon[30138]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:41:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:12 smithi100 ceph-mon[30138]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T17:41:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:12 smithi100 ceph-mon[30138]: Reconfiguring daemon mon.a on smithi100 2024-02-14T17:41:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:12 smithi100 ceph-mon[39557]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:41:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:12 smithi100 ceph-mon[39557]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:41:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:12 smithi100 ceph-mon[39557]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T17:41:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:12 smithi100 ceph-mon[39557]: Reconfiguring daemon mon.a on smithi100 2024-02-14T17:41:12.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:12 smithi179 ceph-mon[33633]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:41:12.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:12 smithi179 ceph-mon[33633]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:41:12.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:12 smithi179 ceph-mon[33633]: Reconfiguring mon.a (unknown last config time)... 2024-02-14T17:41:12.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:12 smithi179 ceph-mon[33633]: Reconfiguring daemon mon.a on smithi100 2024-02-14T17:41:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.442+0000 7fb986a61700 1 -- 172.21.15.100:0/1857972946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb980074010 msgr2=0x7fb9800743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.442+0000 7fb986a61700 1 --2- 172.21.15.100:0/1857972946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb980074010 0x7fb9800743f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb970009910 tx=0x7fb970009c20 comp rx=0 tx=0).stop 2024-02-14T17:41:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.442+0000 7fb986a61700 1 -- 172.21.15.100:0/1857972946 shutdown_connections 2024-02-14T17:41:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.442+0000 7fb986a61700 1 --2- 172.21.15.100:0/1857972946 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb980074010 0x7fb9800743f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.442+0000 7fb986a61700 1 -- 172.21.15.100:0/1857972946 >> 172.21.15.100:0/1857972946 conn(0x7fb9800fd140 msgr2=0x7fb9800ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.443+0000 7fb986a61700 1 -- 172.21.15.100:0/1857972946 shutdown_connections 2024-02-14T17:41:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.443+0000 7fb986a61700 1 -- 172.21.15.100:0/1857972946 wait complete. 2024-02-14T17:41:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.443+0000 7fb986a61700 1 Processor -- start 2024-02-14T17:41:12.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.443+0000 7fb986a61700 1 -- start start 2024-02-14T17:41:12.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.443+0000 7fb986a61700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb980074010 0x7fb98019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:12.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.443+0000 7fb986a61700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98019d270 0x7fb9801a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.443+0000 7fb986a61700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9801a1b40 0x7fb9801a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.443+0000 7fb986a61700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98019d960 con 0x7fb98019d270 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.443+0000 7fb986a61700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9801a98b0 con 0x7fb9801a1b40 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.443+0000 7fb986a61700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9801a99f0 con 0x7fb980074010 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97ffff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb980074010 0x7fb98019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98019d270 0x7fb9801a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97f7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98019d270 0x7fb9801a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47348/0 (socket says 172.21.15.100:47348) 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97f7fe700 1 -- 172.21.15.100:0/483546973 learned_addr learned my addr 172.21.15.100:0/483546973 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97ffff700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb980074010 0x7fb98019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:34932/0 (socket says 172.21.15.100:34932) 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb984ffe700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9801a1b40 0x7fb9801a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97f7fe700 1 -- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb980074010 msgr2=0x7fb98019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97f7fe700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb980074010 0x7fb98019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97f7fe700 1 -- 172.21.15.100:0/483546973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9801a1b40 msgr2=0x7fb9801a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:12.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97f7fe700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9801a1b40 0x7fb9801a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97f7fe700 1 -- 172.21.15.100:0/483546973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9700095f0 con 0x7fb98019d270 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb984ffe700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9801a1b40 0x7fb9801a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97f7fe700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98019d270 0x7fb9801a1600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb97400b780 tx=0x7fb97400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97d7fa700 1 -- 172.21.15.100:0/483546973 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974004230 con 0x7fb98019d270 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97d7fa700 1 -- 172.21.15.100:0/483546973 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb974004390 con 0x7fb98019d270 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb97d7fa700 1 -- 172.21.15.100:0/483546973 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9740119d0 con 0x7fb98019d270 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb986a61700 1 -- 172.21.15.100:0/483546973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9801a9cd0 con 0x7fb98019d270 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.444+0000 7fb986a61700 1 -- 172.21.15.100:0/483546973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9801aa180 con 0x7fb98019d270 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.445+0000 7fb97d7fa700 1 -- 172.21.15.100:0/483546973 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7fb974011b30 con 0x7fb98019d270 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.445+0000 7fb986a61700 1 -- 172.21.15.100:0/483546973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9800632a0 con 0x7fb98019d270 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.445+0000 7fb97d7fa700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fb968039410 0x7fb96803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.445+0000 7fb97d7fa700 1 -- 172.21.15.100:0/483546973 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb97404b660 con 0x7fb98019d270 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.446+0000 7fb97ffff700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fb968039410 0x7fb96803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:12.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.446+0000 7fb97ffff700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fb968039410 0x7fb96803b8d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb970006010 tx=0x7fb970005100 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:12.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.450+0000 7fb97d7fa700 1 -- 172.21.15.100:0/483546973 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb97401a400 con 0x7fb98019d270 2024-02-14T17:41:12.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.646+0000 7fb986a61700 1 -- 172.21.15.100:0/483546973 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb98019e090 con 0x7fb98019d270 2024-02-14T17:41:12.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.646+0000 7fb97d7fa700 1 -- 172.21.15.100:0/483546973 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7fb974018350 con 0x7fb98019d270 2024-02-14T17:41:12.648 INFO:teuthology.orchestra.run.smithi100.stdout:# minimal ceph.conf for d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:41:12.648 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-02-14T17:41:12.648 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:41:12.648 INFO:teuthology.orchestra.run.smithi100.stdout: mon_host = [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] 2024-02-14T17:41:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.648+0000 7fb986a61700 1 -- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fb968039410 msgr2=0x7fb96803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.648+0000 7fb986a61700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fb968039410 0x7fb96803b8d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb970006010 tx=0x7fb970005100 comp rx=0 tx=0).stop 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.648+0000 7fb986a61700 1 -- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98019d270 msgr2=0x7fb9801a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.648+0000 7fb986a61700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98019d270 0x7fb9801a1600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb97400b780 tx=0x7fb97400ba90 comp rx=0 tx=0).stop 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.649+0000 7fb986a61700 1 -- 172.21.15.100:0/483546973 shutdown_connections 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.649+0000 7fb986a61700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fb968039410 0x7fb96803b8d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.649+0000 7fb986a61700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9801a1b40 0x7fb9801a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.649+0000 7fb986a61700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb980074010 0x7fb98019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.649+0000 7fb986a61700 1 --2- 172.21.15.100:0/483546973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb98019d270 0x7fb9801a1600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.649+0000 7fb986a61700 1 -- 172.21.15.100:0/483546973 >> 172.21.15.100:0/483546973 conn(0x7fb9800fd140 msgr2=0x7fb9800fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.649+0000 7fb986a61700 1 -- 172.21.15.100:0/483546973 shutdown_connections 2024-02-14T17:41:12.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:12.649+0000 7fb986a61700 1 -- 172.21.15.100:0/483546973 wait complete. 2024-02-14T17:41:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:13 smithi179 ceph-mon[33633]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/483546973' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:13 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:13 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:13 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:13 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:41:13.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:13 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:13.366 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[39557]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:13.366 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/483546973' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:13.366 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:13.366 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:13.366 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:13.366 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:41:13.366 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:13.366 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[30138]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:13.366 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/483546973' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:13.366 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:13.367 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:13.367 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:13.367 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:41:13.367 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:13 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:13.392 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-14T17:41:13.392 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:41:13.392 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-02-14T17:41:13.423 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:41:13.424 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:41:13.491 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:41:13.491 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.conf 2024-02-14T17:41:13.526 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:41:13.526 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:41:13.597 INFO:tasks.cephadm:Adding mgr.y on smithi100 2024-02-14T17:41:13.597 INFO:tasks.cephadm:Adding mgr.x on smithi179 2024-02-14T17:41:13.597 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply mgr '2;smithi100=y;smithi179=x' 2024-02-14T17:41:13.871 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:41:14.280 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:14 smithi100 ceph-mon[30138]: Reconfiguring mon.c (monmap changed)... 2024-02-14T17:41:14.280 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:14 smithi100 ceph-mon[30138]: Reconfiguring daemon mon.c on smithi100 2024-02-14T17:41:14.280 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:14 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:14.281 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:14 smithi100 ceph-mon[39557]: Reconfiguring mon.c (monmap changed)... 2024-02-14T17:41:14.281 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:14 smithi100 ceph-mon[39557]: Reconfiguring daemon mon.c on smithi100 2024-02-14T17:41:14.281 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:14 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:14.304 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:14 smithi179 ceph-mon[33633]: Reconfiguring mon.c (monmap changed)... 2024-02-14T17:41:14.304 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:14 smithi179 ceph-mon[33633]: Reconfiguring daemon mon.c on smithi100 2024-02-14T17:41:14.304 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:14 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:15.113 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:15 smithi179 ceph-mon[33633]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:15.113 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:15.113 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:15.113 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:15.113 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:41:15.113 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:15.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.331+0000 7f325a947700 1 -- 172.21.15.179:0/2347208185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32541006d0 msgr2=0x7f3254100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:15.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.331+0000 7f325a947700 1 --2- 172.21.15.179:0/2347208185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32541006d0 0x7f3254100af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3244009910 tx=0x7f3244009c20 comp rx=0 tx=0).stop 2024-02-14T17:41:15.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.331+0000 7f325a947700 1 -- 172.21.15.179:0/2347208185 shutdown_connections 2024-02-14T17:41:15.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.331+0000 7f325a947700 1 --2- 172.21.15.179:0/2347208185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32541006d0 0x7f3254100af0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:15.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.331+0000 7f325a947700 1 -- 172.21.15.179:0/2347208185 >> 172.21.15.179:0/2347208185 conn(0x7f32540fbc50 msgr2=0x7f32540fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:15.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.331+0000 7f325a947700 1 -- 172.21.15.179:0/2347208185 shutdown_connections 2024-02-14T17:41:15.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.331+0000 7f325a947700 1 -- 172.21.15.179:0/2347208185 wait complete. 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.331+0000 7f325a947700 1 Processor -- start 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.331+0000 7f325a947700 1 -- start start 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f325a947700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32541006d0 0x7f3254195250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f325a947700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3254195790 0x7f325419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f325a947700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3254195c10 0x7f325419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f325a947700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325419d530 con 0x7f32541006d0 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f325a947700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325419d6a0 con 0x7f3254195c10 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f325a947700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325419d860 con 0x7f3254195790 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f3258ee4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3254195c10 0x7f325419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f3258ee4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3254195c10 0x7f325419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:45140/0 (socket says 172.21.15.179:45140) 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f3258ee4700 1 -- 172.21.15.179:0/992534860 learned_addr learned my addr 172.21.15.179:0/992534860 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f3253fff700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32541006d0 0x7f3254195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f32537fe700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3254195790 0x7f325419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:15.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f3258ee4700 1 -- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3254195790 msgr2=0x7f325419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:15.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f3258ee4700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3254195790 0x7f325419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:15.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f3258ee4700 1 -- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32541006d0 msgr2=0x7f3254195250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:15.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f3258ee4700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32541006d0 0x7f3254195250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:15.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.332+0000 7f3258ee4700 1 -- 172.21.15.179:0/992534860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440095f0 con 0x7f3254195c10 2024-02-14T17:41:15.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.333+0000 7f3258ee4700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3254195c10 0x7f325419cd90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f324c00ba50 tx=0x7f324c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:15.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.333+0000 7f32517fa700 1 -- 172.21.15.179:0/992534860 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c00c930 con 0x7f3254195c10 2024-02-14T17:41:15.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.333+0000 7f32517fa700 1 -- 172.21.15.179:0/992534860 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f324c014440 con 0x7f3254195c10 2024-02-14T17:41:15.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.333+0000 7f325a947700 1 -- 172.21.15.179:0/992534860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325419db40 con 0x7f3254195c10 2024-02-14T17:41:15.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.333+0000 7f32517fa700 1 -- 172.21.15.179:0/992534860 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c0075b0 con 0x7f3254195c10 2024-02-14T17:41:15.337 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.333+0000 7f325a947700 1 -- 172.21.15.179:0/992534860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325419e040 con 0x7f3254195c10 2024-02-14T17:41:15.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.334+0000 7f32517fa700 1 -- 172.21.15.179:0/992534860 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f324c007710 con 0x7f3254195c10 2024-02-14T17:41:15.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.334+0000 7f325a947700 1 -- 172.21.15.179:0/992534860 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32540632a0 con 0x7f3254195c10 2024-02-14T17:41:15.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.334+0000 7f32517fa700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f323c039410 0x7f323c03b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:15.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.334+0000 7f32517fa700 1 -- 172.21.15.179:0/992534860 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f324c04b2d0 con 0x7f3254195c10 2024-02-14T17:41:15.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.334+0000 7f3253fff700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f323c039410 0x7f323c03b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:15.338 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.335+0000 7f3253fff700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f323c039410 0x7f323c03b8d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3244009fd0 tx=0x7f32440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:15.341 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.339+0000 7f32517fa700 1 -- 172.21.15.179:0/992534860 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f324c01b8b0 con 0x7f3254195c10 2024-02-14T17:41:15.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[39557]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[30138]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-14T17:41:15.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:15.554 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.553+0000 7f325a947700 1 -- 172.21.15.179:0/992534860 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi179=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f325418ea40 con 0x7f323c039410 2024-02-14T17:41:15.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.559+0000 7f32517fa700 1 -- 172.21.15.179:0/992534860 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f325418ea40 con 0x7f323c039410 2024-02-14T17:41:15.561 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mgr update... 2024-02-14T17:41:15.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 -- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f323c039410 msgr2=0x7f323c03b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:15.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f323c039410 0x7f323c03b8d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3244009fd0 tx=0x7f32440058e0 comp rx=0 tx=0).stop 2024-02-14T17:41:15.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 -- 172.21.15.179:0/992534860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3254195c10 msgr2=0x7f325419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:15.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3254195c10 0x7f325419cd90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f324c00ba50 tx=0x7f324c00bd60 comp rx=0 tx=0).stop 2024-02-14T17:41:15.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 -- 172.21.15.179:0/992534860 shutdown_connections 2024-02-14T17:41:15.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f323c039410 0x7f323c03b8d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:15.564 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3254195c10 0x7f325419cd90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:15.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32541006d0 0x7f3254195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:15.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 --2- 172.21.15.179:0/992534860 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f3254195790 0x7f325419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:15.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 -- 172.21.15.179:0/992534860 >> 172.21.15.179:0/992534860 conn(0x7f32540fbc50 msgr2=0x7f32540fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:15.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 -- 172.21.15.179:0/992534860 shutdown_connections 2024-02-14T17:41:15.565 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:41:15.562+0000 7f325a947700 1 -- 172.21.15.179:0/992534860 wait complete. 2024-02-14T17:41:16.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:16 smithi100 ceph-mon[30138]: Reconfiguring mon.b (monmap changed)... 2024-02-14T17:41:16.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:16 smithi100 ceph-mon[30138]: Reconfiguring daemon mon.b on smithi179 2024-02-14T17:41:16.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:16 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:16.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:16 smithi100 ceph-mon[39557]: Reconfiguring mon.b (monmap changed)... 2024-02-14T17:41:16.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:16 smithi100 ceph-mon[39557]: Reconfiguring daemon mon.b on smithi179 2024-02-14T17:41:16.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:16 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:16.408 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:16 smithi179 ceph-mon[33633]: Reconfiguring mon.b (monmap changed)... 2024-02-14T17:41:16.409 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:16 smithi179 ceph-mon[33633]: Reconfiguring daemon mon.b on smithi179 2024-02-14T17:41:16.409 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:16 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:16.807 DEBUG:teuthology.orchestra.run.smithi179:mgr.x> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.x.service 2024-02-14T17:41:16.809 INFO:tasks.cephadm:Deploying OSDs... 2024-02-14T17:41:16.809 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:41:16.809 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:41:16.828 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-14T17:41:16.829 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-02-14T17:41:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-14T17:41:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:41:16.887 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-14T17:41:16.887 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:41:16.887 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:41:16.887 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-02-14 17:41:02.866862323 +0000 2024-02-14T17:41:16.887 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-02-14 17:34:56.915993809 +0000 2024-02-14T17:41:16.887 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-02-14 17:34:56.915993809 +0000 2024-02-14T17:41:16.887 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-02-14 17:34:56.915993809 +0000 2024-02-14T17:41:16.888 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-14T17:41:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-02-14T17:41:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-02-14T17:41:16.958 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000248095 s, 2.1 MB/s 2024-02-14T17:41:16.959 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-14T17:41:17.018 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-02-14T17:41:17.075 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-14T17:41:17.075 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:41:17.075 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-14T17:41:17.075 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:41:17.075 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:41:17.075 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-02-14 17:41:02.867862298 +0000 2024-02-14T17:41:17.075 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-02-14 17:34:57.262984882 +0000 2024-02-14T17:41:17.075 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-02-14 17:34:57.262984882 +0000 2024-02-14T17:41:17.076 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-02-14 17:34:57.262984882 +0000 2024-02-14T17:41:17.076 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-14T17:41:17.143 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-02-14T17:41:17.144 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-02-14T17:41:17.144 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000222089 s, 2.3 MB/s 2024-02-14T17:41:17.145 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-14T17:41:17.203 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-02-14T17:41:17.258 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-14T17:41:17.258 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:41:17.258 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-14T17:41:17.258 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:41:17.258 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:41:17.259 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-02-14 17:41:02.867862298 +0000 2024-02-14T17:41:17.259 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-02-14 17:34:57.595976316 +0000 2024-02-14T17:41:17.259 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-02-14 17:34:57.595976316 +0000 2024-02-14T17:41:17.259 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-02-14 17:34:57.595976316 +0000 2024-02-14T17:41:17.259 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-14T17:41:17.323 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi179=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:17.323 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: Saving service mgr spec with placement smithi100=y;smithi179=x;count:2 2024-02-14T17:41:17.323 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:17.323 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi179=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: Saving service mgr spec with placement smithi100=y;smithi179=x;count:2 2024-02-14T17:41:17.324 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:17.325 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:17.325 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:17.325 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:17.325 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:17.325 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:17.325 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:17.325 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:41:17.325 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:41:17.325 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:41:17.325 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:17.325 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-02-14T17:41:17.326 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-02-14T17:41:17.326 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000197834 s, 2.6 MB/s 2024-02-14T17:41:17.327 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-14T17:41:17.384 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi100=y;smithi179=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: Saving service mgr spec with placement smithi100=y;smithi179=x;count:2 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-14T17:41:17.389 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:41:17.390 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:17.439 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-14T17:41:17.440 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:41:17.440 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-14T17:41:17.440 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:41:17.440 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:41:17.440 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-02-14 17:41:02.868862272 +0000 2024-02-14T17:41:17.440 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-02-14 17:34:57.942967389 +0000 2024-02-14T17:41:17.440 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-02-14 17:34:57.942967389 +0000 2024-02-14T17:41:17.440 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-02-14 17:34:57.942967389 +0000 2024-02-14T17:41:17.440 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-14T17:41:17.506 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-02-14T17:41:17.506 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-02-14T17:41:17.506 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000193592 s, 2.6 MB/s 2024-02-14T17:41:17.507 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-14T17:41:17.564 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:41:17.564 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-02-14T17:41:17.580 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-14T17:41:17.580 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-02-14T17:41:17.635 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-14T17:41:17.635 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:41:17.635 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-14T17:41:17.635 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:41:17.635 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:41:17.635 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-02-14 17:40:53.429372950 +0000 2024-02-14T17:41:17.635 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-02-14 17:34:56.923278790 +0000 2024-02-14T17:41:17.635 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-02-14 17:34:56.923278790 +0000 2024-02-14T17:41:17.635 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-02-14 17:34:56.923278790 +0000 2024-02-14T17:41:17.636 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-14T17:41:17.704 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-02-14T17:41:17.704 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-02-14T17:41:17.704 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000211477 s, 2.4 MB/s 2024-02-14T17:41:17.705 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-14T17:41:17.763 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-02-14T17:41:17.818 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-14T17:41:17.818 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:41:17.818 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-14T17:41:17.819 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:41:17.819 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:41:17.819 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-02-14 17:40:53.429372950 +0000 2024-02-14T17:41:17.819 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-02-14 17:34:57.266271135 +0000 2024-02-14T17:41:17.819 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-02-14 17:34:57.266271135 +0000 2024-02-14T17:41:17.819 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-02-14 17:34:57.266271135 +0000 2024-02-14T17:41:17.819 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-14T17:41:17.885 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-02-14T17:41:17.885 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-02-14T17:41:17.885 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000194088 s, 2.6 MB/s 2024-02-14T17:41:17.886 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-14T17:41:17.941 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-02-14T17:41:17.996 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-14T17:41:17.996 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:41:17.996 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-14T17:41:17.996 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:41:17.996 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:41:17.996 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-02-14 17:40:53.429372950 +0000 2024-02-14T17:41:17.996 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-02-14 17:34:57.617263303 +0000 2024-02-14T17:41:17.996 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-02-14 17:34:57.617263303 +0000 2024-02-14T17:41:17.996 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-02-14 17:34:57.617263303 +0000 2024-02-14T17:41:17.997 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-14T17:41:18.062 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-02-14T17:41:18.062 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-02-14T17:41:18.062 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000167924 s, 3.0 MB/s 2024-02-14T17:41:18.064 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-14T17:41:18.119 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-02-14T17:41:18.174 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-14T17:41:18.174 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-14T17:41:18.174 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-14T17:41:18.175 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-14T17:41:18.175 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-02-14T17:41:18.175 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-02-14 17:40:53.430372927 +0000 2024-02-14T17:41:18.175 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-02-14 17:34:57.994254890 +0000 2024-02-14T17:41:18.175 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-02-14 17:34:57.994254890 +0000 2024-02-14T17:41:18.175 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-02-14 17:34:57.994254890 +0000 2024-02-14T17:41:18.175 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-14T17:41:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:18 smithi179 ceph-mon[33633]: Deploying daemon mgr.x on smithi179 2024-02-14T17:41:18.370 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-02-14T17:41:18.370 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-02-14T17:41:18.370 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000171759 s, 3.0 MB/s 2024-02-14T17:41:18.371 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-14T17:41:18.386 INFO:tasks.cephadm:Deploying osd.0 on smithi100 with /dev/vg_nvme/lv_4... 2024-02-14T17:41:18.386 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-14T17:41:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:18 smithi100 ceph-mon[30138]: Deploying daemon mgr.x on smithi179 2024-02-14T17:41:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:18 smithi100 ceph-mon[39557]: Deploying daemon mgr.x on smithi179 2024-02-14T17:41:18.632 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:41:18.786 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:18 smithi179 systemd[1]: Starting Ceph mgr.x for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:41:19.072 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:18 smithi179 podman[35700]: 2024-02-14 17:41:18.687010494 +0000 UTC m=+0.017521395 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:41:19.072 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:18 smithi179 podman[35700]: 2024-02-14 17:41:18.819669302 +0000 UTC m=+0.150180234 container create 3f087bfacba1634488f736d88895c59dd3f558925e1c5c9defd83494a42b03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-14T17:41:19.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:19 smithi100 ceph-mon[30138]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:19.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:19 smithi100 ceph-mon[39557]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:19 smithi179 ceph-mon[33633]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:19.572 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:19 smithi179 podman[35700]: 2024-02-14 17:41:19.12201186 +0000 UTC m=+0.452522751 container init 3f087bfacba1634488f736d88895c59dd3f558925e1c5c9defd83494a42b03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-14T17:41:19.572 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:19 smithi179 podman[35700]: 2024-02-14 17:41:19.127514231 +0000 UTC m=+0.458025122 container start 3f087bfacba1634488f736d88895c59dd3f558925e1c5c9defd83494a42b03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True) 2024-02-14T17:41:19.572 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:19 smithi179 ceph-mgr[35738]: -- 172.21.15.179:0/3016090365 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561818a8c1a0 con 0x561818a63000 2024-02-14T17:41:19.572 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:19 smithi179 bash[35700]: 3f087bfacba1634488f736d88895c59dd3f558925e1c5c9defd83494a42b03ef 2024-02-14T17:41:19.572 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:19 smithi179 systemd[1]: Started Ceph mgr.x for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:41:20.071 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:19 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:19.686+0000 7f193dc8d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:41:20.071 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:19 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:19.783+0000 7f193dc8d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:41:20.571 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:20.483+0000 7f193dc8d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:41:20.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:20.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-mon[33633]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:20.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:20.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[30138]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:20.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:20.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:20.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:20 smithi100 ceph-mon[39557]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:20.829 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-14T17:41:20.829 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-14T17:41:20.829 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-02-14T17:41:20.829 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-02-14T17:41:20.829 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0145548 s, 720 MB/s 2024-02-14T17:41:20.829 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-02-14T17:41:20.834 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:20.580+0000 7f193dc8d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:41:20.834 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:20.734+0000 7f193dc8d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:41:20.834 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:20.832+0000 7f193dc8d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:41:21.179 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:20.915+0000 7f193dc8d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:41:21.179 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:21.177+0000 7f193dc8d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:41:21.468 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:21.286+0000 7f193dc8d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:41:21.468 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:21.372+0000 7f193dc8d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:41:21.468 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:21.466+0000 7f193dc8d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:41:21.470 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-02-14T17:41:21.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:21.696 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[30138]: Reconfiguring mgr.y (unknown last config time)... 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[30138]: Reconfiguring daemon mgr.y on smithi100 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:21.697 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:21.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:21.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[39557]: Reconfiguring mgr.y (unknown last config time)... 2024-02-14T17:41:21.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:41:21.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:41:21.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:21.698 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:21 smithi100 ceph-mon[39557]: Reconfiguring daemon mgr.y on smithi100 2024-02-14T17:41:21.821 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:21.641+0000 7f193dc8d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:41:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-mon[33633]: Reconfiguring mgr.y (unknown last config time)... 2024-02-14T17:41:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-14T17:41:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-14T17:41:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:21 smithi179 ceph-mon[33633]: Reconfiguring daemon mgr.y on smithi100 2024-02-14T17:41:22.299 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:41:23.046 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:22 smithi100 ceph-mon[30138]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:23.046 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:22 smithi100 ceph-mon[39557]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:23.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:22 smithi179 ceph-mon[33633]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:23.071 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:22.605+0000 7f193dc8d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:41:23.071 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:22.733+0000 7f193dc8d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:41:23.361 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:23.078+0000 7f193dc8d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:41:23.361 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:23.167+0000 7f193dc8d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:41:23.361 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:23.359+0000 7f193dc8d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:41:23.821 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:23.539+0000 7f193dc8d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:41:24.087 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.087 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.087 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:24.087 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.087 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:24.087 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:24.087 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.087 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:23.822+0000 7f193dc8d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:41:24.087 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:24.085+0000 7f193dc8d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:41:24.125 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.125 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.125 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:24.125 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.125 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:24.125 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:24.125 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:24.126 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.126 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:24.126 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:24.126 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:23 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:24.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.355+0000 7f5c82b6f700 1 -- 172.21.15.100:0/2845485185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c102740 msgr2=0x7f5c7c108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:24.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.355+0000 7f5c82b6f700 1 --2- 172.21.15.100:0/2845485185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c102740 0x7f5c7c108ff0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009fc0 tx=0x7f5c7400b330 comp rx=0 tx=0).stop 2024-02-14T17:41:24.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.355+0000 7f5c82b6f700 1 -- 172.21.15.100:0/2845485185 shutdown_connections 2024-02-14T17:41:24.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.355+0000 7f5c82b6f700 1 --2- 172.21.15.100:0/2845485185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c102740 0x7f5c7c108ff0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.355+0000 7f5c82b6f700 1 --2- 172.21.15.100:0/2845485185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c7c100ba0 0x7f5c7c100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.355+0000 7f5c82b6f700 1 --2- 172.21.15.100:0/2845485185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c7c101da0 0x7f5c7c102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.355+0000 7f5c82b6f700 1 -- 172.21.15.100:0/2845485185 >> 172.21.15.100:0/2845485185 conn(0x7f5c7c0fc160 msgr2=0x7f5c7c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.355+0000 7f5c82b6f700 1 -- 172.21.15.100:0/2845485185 shutdown_connections 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.355+0000 7f5c82b6f700 1 -- 172.21.15.100:0/2845485185 wait complete. 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c82b6f700 1 Processor -- start 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c82b6f700 1 -- start start 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c82b6f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c7c100ba0 0x7f5c7c104b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c82b6f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c101da0 0x7f5c7c103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c82b6f700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c7c102740 0x7f5c7c1036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:24.361 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c82b6f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c103e40 con 0x7f5c7c101da0 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c82b6f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c103fb0 con 0x7f5c7c100ba0 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c82b6f700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1a1350 con 0x7f5c7c102740 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c8110c700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c7c102740 0x7f5c7c1036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c8110c700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c7c102740 0x7f5c7c1036c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:48132/0 (socket says 172.21.15.100:48132) 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c8110c700 1 -- 172.21.15.100:0/981540715 learned_addr learned my addr 172.21.15.100:0/981540715 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c8090b700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c7c100ba0 0x7f5c7c104b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c7bfff700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c101da0 0x7f5c7c103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c8110c700 1 -- 172.21.15.100:0/981540715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c7c100ba0 msgr2=0x7f5c7c104b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c8110c700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c7c100ba0 0x7f5c7c104b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c8110c700 1 -- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c101da0 msgr2=0x7f5c7c103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c8110c700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c101da0 0x7f5c7c103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.356+0000 7f5c8110c700 1 -- 172.21.15.100:0/981540715 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700095f0 con 0x7f5c7c102740 2024-02-14T17:41:24.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.357+0000 7f5c7bfff700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c101da0 0x7f5c7c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.357+0000 7f5c8110c700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c7c102740 0x7f5c7c1036c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c740079d0 tx=0x7f5c74007a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.357+0000 7f5c79ffb700 1 -- 172.21.15.100:0/981540715 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400e040 con 0x7f5c7c102740 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.357+0000 7f5c82b6f700 1 -- 172.21.15.100:0/981540715 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c74009c40 con 0x7f5c7c102740 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.357+0000 7f5c79ffb700 1 -- 172.21.15.100:0/981540715 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5c7400be70 con 0x7f5c7c102740 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.357+0000 7f5c79ffb700 1 -- 172.21.15.100:0/981540715 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c740056d0 con 0x7f5c7c102740 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.357+0000 7f5c82b6f700 1 -- 172.21.15.100:0/981540715 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c197240 con 0x7f5c7c102740 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.358+0000 7f5c79ffb700 1 -- 172.21.15.100:0/981540715 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 13) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f5c74005830 con 0x7f5c7c102740 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.358+0000 7f5c82b6f700 1 -- 172.21.15.100:0/981540715 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c108980 con 0x7f5c7c102740 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.358+0000 7f5c79ffb700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f5c6c0391c0 0x7f5c6c03b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.358+0000 7f5c79ffb700 1 -- 172.21.15.100:0/981540715 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5c740551a0 con 0x7f5c7c102740 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.358+0000 7f5c8090b700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f5c6c0391c0 0x7f5c6c03b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:24.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.359+0000 7f5c8090b700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f5c6c0391c0 0x7f5c6c03b680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c101c00 tx=0x7f5c70009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:24.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.363+0000 7f5c79ffb700 1 -- 172.21.15.100:0/981540715 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c74012e30 con 0x7f5c7c102740 2024-02-14T17:41:24.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:24.564+0000 7f5c82b6f700 1 -- 172.21.15.100:0/981540715 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c7c0620b0 con 0x7f5c6c0391c0 2024-02-14T17:41:24.571 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:24.319+0000 7f193dc8d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:41:25.285 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:24 smithi100 ceph-mon[30138]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:25.285 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:24 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:41:25.285 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:24 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:41:25.285 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:24 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:25.285 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:24 smithi100 ceph-mon[39557]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:25.285 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:24 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:41:25.285 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:24 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:41:25.285 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:24 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:24 smithi179 ceph-mon[33633]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:24 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:41:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:24 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:41:25.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:24 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:25.322 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:25 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:25.112+0000 7f193dc8d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:41:25.962 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:25 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:25.677+0000 7f193dc8d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:41:25.962 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:25 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:25.773+0000 7f193dc8d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:41:26.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:25 smithi100 ceph-mon[30138]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:26.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:25 smithi100 ceph-mon[39557]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:25 smithi179 ceph-mon[33633]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:26.322 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:41:25 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:41:25.987+0000 7f193dc8d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:41:26.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:26.969+0000 7f5c79ffb700 1 -- 172.21.15.100:0/981540715 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mgrmap(e 14) v1 ==== 91351+0+0 (secure 0 0 0) 0x7f5c7402bda0 con 0x7f5c7c102740 2024-02-14T17:41:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:26 smithi179 ceph-mon[33633]: Standby manager daemon x started 2024-02-14T17:41:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:26 smithi179 ceph-mon[33633]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:41:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:26 smithi179 ceph-mon[33633]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:41:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:26 smithi179 ceph-mon[33633]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:41:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:26 smithi179 ceph-mon[33633]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:41:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:26 smithi179 ceph-mon[33633]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[30138]: Standby manager daemon x started 2024-02-14T17:41:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[30138]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:41:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[30138]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:41:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[30138]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:41:27.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[30138]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:41:27.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[30138]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[39557]: Standby manager daemon x started 2024-02-14T17:41:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[39557]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:41:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[39557]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:41:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[39557]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:41:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[39557]: from='mgr.? 172.21.15.179:0/3085259387' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:41:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:26 smithi100 ceph-mon[39557]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:28.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:27 smithi179 ceph-mon[33633]: mgrmap e14: y(active, since 68s), standbys: x 2024-02-14T17:41:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:27 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:41:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3905653200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d822e357-3e1f-407f-a403-8b994ccc1f84"}]: dispatch 2024-02-14T17:41:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d822e357-3e1f-407f-a403-8b994ccc1f84"}]: dispatch 2024-02-14T17:41:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d822e357-3e1f-407f-a403-8b994ccc1f84"}]': finished 2024-02-14T17:41:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:27 smithi179 ceph-mon[33633]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T17:41:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:27 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1652164865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:28.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[30138]: mgrmap e14: y(active, since 68s), standbys: x 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3905653200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d822e357-3e1f-407f-a403-8b994ccc1f84"}]: dispatch 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d822e357-3e1f-407f-a403-8b994ccc1f84"}]: dispatch 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d822e357-3e1f-407f-a403-8b994ccc1f84"}]': finished 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[30138]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1652164865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[39557]: mgrmap e14: y(active, since 68s), standbys: x 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3905653200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d822e357-3e1f-407f-a403-8b994ccc1f84"}]: dispatch 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d822e357-3e1f-407f-a403-8b994ccc1f84"}]: dispatch 2024-02-14T17:41:28.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d822e357-3e1f-407f-a403-8b994ccc1f84"}]': finished 2024-02-14T17:41:28.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[39557]: osdmap e5: 1 total, 0 up, 1 in 2024-02-14T17:41:28.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:28.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1652164865' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:29.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:28 smithi179 ceph-mon[33633]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:28 smithi100 ceph-mon[30138]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:29.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:28 smithi100 ceph-mon[39557]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:31 smithi100 ceph-mon[30138]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:31 smithi100 ceph-mon[39557]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:31 smithi179 ceph-mon[33633]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:33 smithi100 ceph-mon[30138]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:33 smithi100 ceph-mon[39557]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:33 smithi179 ceph-mon[33633]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T17:41:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:34.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:34 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T17:41:34.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:34 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:34 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-14T17:41:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:34 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:35 smithi100 ceph-mon[30138]: Deploying daemon osd.0 on smithi100 2024-02-14T17:41:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:35 smithi100 ceph-mon[30138]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:35 smithi100 ceph-mon[39557]: Deploying daemon osd.0 on smithi100 2024-02-14T17:41:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:35 smithi100 ceph-mon[39557]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:35 smithi179 ceph-mon[33633]: Deploying daemon osd.0 on smithi100 2024-02-14T17:41:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:35 smithi179 ceph-mon[33633]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:37.345 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:37 smithi100 ceph-mon[30138]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:37.346 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:37 smithi100 ceph-mon[39557]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:37 smithi179 ceph-mon[33633]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:39 smithi100 ceph-mon[30138]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:39.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:39 smithi100 ceph-mon[39557]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:39 smithi179 ceph-mon[33633]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:40.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:40.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:40.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:40.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:40 smithi100 ceph-mon[30138]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:40.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:40.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:40.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:40.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:40 smithi100 ceph-mon[39557]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:40 smithi179 ceph-mon[33633]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:43.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[30138]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:43.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:43.201 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:43.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:43.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:43.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:43.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[30138]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:41:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[39557]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:43.203 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:43.204 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:43 smithi100 ceph-mon[39557]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:41:43.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.267+0000 7f5c79ffb700 1 -- 172.21.15.100:0/981540715 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5c7c0620b0 con 0x7f5c6c0391c0 2024-02-14T17:41:43.270 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 0 on host 'smithi100' 2024-02-14T17:41:43.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 -- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f5c6c0391c0 msgr2=0x7f5c6c03b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:43.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f5c6c0391c0 0x7f5c6c03b680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c101c00 tx=0x7f5c70009310 comp rx=0 tx=0).stop 2024-02-14T17:41:43.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 -- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c7c102740 msgr2=0x7f5c7c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:43.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c7c102740 0x7f5c7c1036c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c740079d0 tx=0x7f5c74007a00 comp rx=0 tx=0).stop 2024-02-14T17:41:43.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 -- 172.21.15.100:0/981540715 shutdown_connections 2024-02-14T17:41:43.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f5c6c0391c0 0x7f5c6c03b680 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:43.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c7c100ba0 0x7f5c7c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:43.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c7c102740 0x7f5c7c1036c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:43.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 --2- 172.21.15.100:0/981540715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c7c101da0 0x7f5c7c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:43.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 -- 172.21.15.100:0/981540715 >> 172.21.15.100:0/981540715 conn(0x7f5c7c0fc160 msgr2=0x7f5c7c06ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:43.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 -- 172.21.15.100:0/981540715 shutdown_connections 2024-02-14T17:41:43.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:43.270+0000 7f5c82b6f700 1 -- 172.21.15.100:0/981540715 wait complete. 2024-02-14T17:41:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:43 smithi179 ceph-mon[33633]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:43 smithi179 ceph-mon[33633]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-14T17:41:44.017 DEBUG:teuthology.orchestra.run.smithi100:osd.0> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.0.service 2024-02-14T17:41:44.020 INFO:tasks.cephadm:Deploying osd.1 on smithi100 with /dev/vg_nvme/lv_3... 2024-02-14T17:41:44.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-14T17:41:44.332 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:44.332 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:44.332 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:44.332 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[39557]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T17:41:44.333 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[39557]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T17:41:44.333 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[39557]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:41:44.333 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:44.333 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:44.333 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:44.333 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:44.333 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[30138]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T17:41:44.333 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[30138]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T17:41:44.333 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[30138]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:41:44.333 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:41:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:44 smithi179 ceph-mon[33633]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-14T17:41:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:44 smithi179 ceph-mon[33633]: osdmap e6: 1 total, 0 up, 1 in 2024-02-14T17:41:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:44 smithi179 ceph-mon[33633]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:41:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:44.606 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0[43959]: 2024-02-14T17:41:44.427+0000 7f813ac23700 -1 osd.0 0 waiting for initial osdmap 2024-02-14T17:41:44.606 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:41:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0[43959]: 2024-02-14T17:41:44.431+0000 7f8136a16700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:41:44.832 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:41:45.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:45 smithi100 ceph-mon[39557]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:45.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:45 smithi100 ceph-mon[39557]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:41:45.201 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:45 smithi100 ceph-mon[39557]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T17:41:45.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:45 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:45.202 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:45 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:45.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:45 smithi100 ceph-mon[30138]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:45.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:45 smithi100 ceph-mon[30138]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:41:45.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:45 smithi100 ceph-mon[30138]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T17:41:45.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:45 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:45.202 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:45 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:45 smithi179 ceph-mon[33633]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:45 smithi179 ceph-mon[33633]: from='osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:41:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:45 smithi179 ceph-mon[33633]: osdmap e7: 1 total, 0 up, 1 in 2024-02-14T17:41:45.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:45 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:45.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:45 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:46.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[30138]: purged_snaps scrub starts 2024-02-14T17:41:46.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[30138]: purged_snaps scrub ok 2024-02-14T17:41:46.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:46.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[30138]: osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638] boot 2024-02-14T17:41:46.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[30138]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T17:41:46.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:46.363 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[39557]: purged_snaps scrub starts 2024-02-14T17:41:46.363 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[39557]: purged_snaps scrub ok 2024-02-14T17:41:46.363 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:46.363 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[39557]: osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638] boot 2024-02-14T17:41:46.363 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[39557]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T17:41:46.363 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:46 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:46.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:46 smithi179 ceph-mon[33633]: purged_snaps scrub starts 2024-02-14T17:41:46.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:46 smithi179 ceph-mon[33633]: purged_snaps scrub ok 2024-02-14T17:41:46.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:46 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:46.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:46 smithi179 ceph-mon[33633]: osd.0 [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638] boot 2024-02-14T17:41:46.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:46 smithi179 ceph-mon[33633]: osdmap e8: 1 total, 1 up, 1 in 2024-02-14T17:41:46.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:46 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:41:47.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:47 smithi100 ceph-mon[30138]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:47.284 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:47 smithi100 ceph-mon[30138]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T17:41:47.284 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:47 smithi100 ceph-mon[39557]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:47.284 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:47 smithi100 ceph-mon[39557]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T17:41:47.488 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:47 smithi179 ceph-mon[33633]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-14T17:41:47.488 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:47 smithi179 ceph-mon[33633]: osdmap e9: 1 total, 1 up, 1 in 2024-02-14T17:41:47.740 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-14T17:41:47.741 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-14T17:41:47.741 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-02-14T17:41:47.741 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-02-14T17:41:47.741 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125307 s, 837 MB/s 2024-02-14T17:41:47.741 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-02-14T17:41:48.827 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-02-14T17:41:49.063 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:41:49.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[30138]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:49.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:49.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:49.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:49.348 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.349 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[39557]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:49.349 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.349 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.349 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:49.349 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.349 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:49.349 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:49.349 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:49 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:49 smithi179 ceph-mon[33633]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:49 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:49 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:49 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:41:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:49 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:49 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:49 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:41:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:49 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:41:50.248 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:50 smithi100 ceph-mon[30138]: Detected new or changed devices on smithi100 2024-02-14T17:41:50.248 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:50 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi100 to 16064M 2024-02-14T17:41:50.248 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:50 smithi100 ceph-mon[39557]: Detected new or changed devices on smithi100 2024-02-14T17:41:50.248 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:50 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi100 to 16064M 2024-02-14T17:41:50.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:50 smithi179 ceph-mon[33633]: Detected new or changed devices on smithi100 2024-02-14T17:41:50.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:50 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi100 to 16064M 2024-02-14T17:41:50.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.798+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/721964195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c44103840 msgr2=0x7f6c4410a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.798+0000 7f6c4ae5b700 1 --2- 172.21.15.100:0/721964195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c44103840 0x7f6c4410a0f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00b0a0 tx=0x7f6c3c00b3b0 comp rx=0 tx=0).stop 2024-02-14T17:41:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.798+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/721964195 shutdown_connections 2024-02-14T17:41:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.798+0000 7f6c4ae5b700 1 --2- 172.21.15.100:0/721964195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c44101ca0 0x7f6c441020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.798+0000 7f6c4ae5b700 1 --2- 172.21.15.100:0/721964195 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c44103840 0x7f6c4410a0f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.799+0000 7f6c4ae5b700 1 --2- 172.21.15.100:0/721964195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c44102ea0 0x7f6c44103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:50.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.799+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/721964195 >> 172.21.15.100:0/721964195 conn(0x7f6c440fd280 msgr2=0x7f6c440ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:41:50.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.799+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/721964195 shutdown_connections 2024-02-14T17:41:50.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.799+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/721964195 wait complete. 2024-02-14T17:41:50.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.799+0000 7f6c4ae5b700 1 Processor -- start 2024-02-14T17:41:50.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.799+0000 7f6c4ae5b700 1 -- start start 2024-02-14T17:41:50.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.799+0000 7f6c4ae5b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c44101ca0 0x7f6c441972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:50.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.799+0000 7f6c4ae5b700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c44102ea0 0x7f6c44197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:50.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c4ae5b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c44103840 0x7f6c4419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:50.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c4ae5b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4419ee50 con 0x7f6c44101ca0 2024-02-14T17:41:50.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c4ae5b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4419ef90 con 0x7f6c44103840 2024-02-14T17:41:50.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c4ae5b700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4419f120 con 0x7f6c44102ea0 2024-02-14T17:41:50.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c43fff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c44102ea0 0x7f6c44197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:50.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c48bf7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c44101ca0 0x7f6c441972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:50.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c43fff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c44102ea0 0x7f6c44197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:57776/0 (socket says 172.21.15.100:57776) 2024-02-14T17:41:50.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c48bf7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c44101ca0 0x7f6c441972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:40014/0 (socket says 172.21.15.100:40014) 2024-02-14T17:41:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c43fff700 1 -- 172.21.15.100:0/1904851385 learned_addr learned my addr 172.21.15.100:0/1904851385 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:41:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c493f8700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c44103840 0x7f6c4419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c48bf7700 1 -- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c44102ea0 msgr2=0x7f6c44197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c48bf7700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c44102ea0 0x7f6c44197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c48bf7700 1 -- 172.21.15.100:0/1904851385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c44103840 msgr2=0x7f6c4419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:41:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c48bf7700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c44103840 0x7f6c4419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:41:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c48bf7700 1 -- 172.21.15.100:0/1904851385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30009520 con 0x7f6c44101ca0 2024-02-14T17:41:50.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c48bf7700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c44101ca0 0x7f6c441972c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009ae0 tx=0x7f6c38009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:50.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c41ffb700 1 -- 172.21.15.100:0/1904851385 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38014070 con 0x7f6c44101ca0 2024-02-14T17:41:50.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/1904851385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c009d00 con 0x7f6c44101ca0 2024-02-14T17:41:50.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c41ffb700 1 -- 172.21.15.100:0/1904851385 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6c380044a0 con 0x7f6c44101ca0 2024-02-14T17:41:50.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.800+0000 7f6c41ffb700 1 -- 172.21.15.100:0/1904851385 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38007890 con 0x7f6c44101ca0 2024-02-14T17:41:50.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.801+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/1904851385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c44074270 con 0x7f6c44101ca0 2024-02-14T17:41:50.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.801+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/1904851385 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c44108880 con 0x7f6c44101ca0 2024-02-14T17:41:50.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.802+0000 7f6c41ffb700 1 -- 172.21.15.100:0/1904851385 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 14) v1 ==== 91351+0+0 (secure 0 0 0) 0x7f6c38018070 con 0x7f6c44101ca0 2024-02-14T17:41:50.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.802+0000 7f6c41ffb700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6c3406d940 0x7f6c3406fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:41:50.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.802+0000 7f6c41ffb700 1 -- 172.21.15.100:0/1904851385 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f6c3808ef80 con 0x7f6c44101ca0 2024-02-14T17:41:50.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.802+0000 7f6c43fff700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6c3406d940 0x7f6c3406fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:41:50.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.803+0000 7f6c43fff700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6c3406d940 0x7f6c3406fe00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009ef0 tx=0x7f6c30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:41:50.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:50.807+0000 7f6c41ffb700 1 -- 172.21.15.100:0/1904851385 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c38059ae0 con 0x7f6c44101ca0 2024-02-14T17:41:51.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:41:51.009+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/1904851385 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c4410cdc0 con 0x7f6c3406d940 2024-02-14T17:41:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:51 smithi100 ceph-mon[30138]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:51 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:41:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:51 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:41:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:51 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:51 smithi100 ceph-mon[39557]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:51 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:41:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:51 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:41:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:51 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:51 smithi179 ceph-mon[33633]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:51 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:41:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:51 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:41:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:51 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:41:52.349 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:52 smithi100 ceph-mon[30138]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:52.350 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:52 smithi100 ceph-mon[39557]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:52.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:52 smithi179 ceph-mon[33633]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:41:53.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:53 smithi100 ceph-mon[30138]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:53.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:53 smithi100 ceph-mon[39557]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:53 smithi179 ceph-mon[33633]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/989516203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f74efd5d-9766-407c-9b93-8fc21cf42e1f"}]: dispatch 2024-02-14T17:41:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f74efd5d-9766-407c-9b93-8fc21cf42e1f"}]: dispatch 2024-02-14T17:41:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f74efd5d-9766-407c-9b93-8fc21cf42e1f"}]': finished 2024-02-14T17:41:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:54 smithi100 ceph-mon[30138]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T17:41:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:41:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/989516203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f74efd5d-9766-407c-9b93-8fc21cf42e1f"}]: dispatch 2024-02-14T17:41:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f74efd5d-9766-407c-9b93-8fc21cf42e1f"}]: dispatch 2024-02-14T17:41:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f74efd5d-9766-407c-9b93-8fc21cf42e1f"}]': finished 2024-02-14T17:41:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:54 smithi100 ceph-mon[39557]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T17:41:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:41:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/989516203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f74efd5d-9766-407c-9b93-8fc21cf42e1f"}]: dispatch 2024-02-14T17:41:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f74efd5d-9766-407c-9b93-8fc21cf42e1f"}]: dispatch 2024-02-14T17:41:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f74efd5d-9766-407c-9b93-8fc21cf42e1f"}]': finished 2024-02-14T17:41:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:54 smithi179 ceph-mon[33633]: osdmap e10: 2 total, 1 up, 2 in 2024-02-14T17:41:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:41:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:55 smithi100 ceph-mon[30138]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3650763453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:55.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:55 smithi100 ceph-mon[39557]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:55.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3650763453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:55 smithi179 ceph-mon[33633]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3650763453' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:41:57.275 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:57 smithi100 ceph-mon[30138]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:57.276 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:57 smithi100 ceph-mon[39557]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:57 smithi179 ceph-mon[33633]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:59.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:41:59 smithi100 ceph-mon[30138]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:59.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:41:59 smithi100 ceph-mon[39557]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:41:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:41:59 smithi179 ceph-mon[33633]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:01 smithi100 ceph-mon[39557]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:01.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:01 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T17:42:01.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:01 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:01.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:01 smithi100 ceph-mon[30138]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:01.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:01 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T17:42:01.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:01 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:01 smithi179 ceph-mon[33633]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:01.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:01 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-14T17:42:01.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:01 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:02.373 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:02 smithi100 ceph-mon[30138]: Deploying daemon osd.1 on smithi100 2024-02-14T17:42:02.374 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:02 smithi100 ceph-mon[39557]: Deploying daemon osd.1 on smithi100 2024-02-14T17:42:02.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:02 smithi179 ceph-mon[33633]: Deploying daemon osd.1 on smithi100 2024-02-14T17:42:03.127 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:03 smithi100 ceph-mon[30138]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:03 smithi100 ceph-mon[39557]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:03 smithi179 ceph-mon[33633]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:05.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:05 smithi100 ceph-mon[30138]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:05.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:05 smithi100 ceph-mon[39557]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:05 smithi179 ceph-mon[33633]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:06.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:06 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:06.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:06 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:06.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:06 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:06.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:06 smithi100 ceph-mon[39557]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:06 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:06 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:06 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:06 smithi100 ceph-mon[30138]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:06 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:06 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:06 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:06 smithi179 ceph-mon[33633]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[30138]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[30138]: from='osd.1 [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[30138]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[39557]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:09.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:09.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[39557]: from='osd.1 [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:42:09.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:09 smithi100 ceph-mon[39557]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:42:09.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.514+0000 7f6c41ffb700 1 -- 172.21.15.100:0/1904851385 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6c4410cdc0 con 0x7f6c3406d940 2024-02-14T17:42:09.517 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 1 on host 'smithi100' 2024-02-14T17:42:09.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.516+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6c3406d940 msgr2=0x7f6c3406fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.516+0000 7f6c4ae5b700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6c3406d940 0x7f6c3406fe00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009ef0 tx=0x7f6c30009260 comp rx=0 tx=0).stop 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.516+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c44101ca0 msgr2=0x7f6c441972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.516+0000 7f6c4ae5b700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c44101ca0 0x7f6c441972c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009ae0 tx=0x7f6c38009df0 comp rx=0 tx=0).stop 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.516+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/1904851385 shutdown_connections 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.517+0000 7f6c4ae5b700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6c3406d940 0x7f6c3406fe00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.517+0000 7f6c4ae5b700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c44103840 0x7f6c4419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.517+0000 7f6c4ae5b700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c44101ca0 0x7f6c441972c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.517+0000 7f6c4ae5b700 1 --2- 172.21.15.100:0/1904851385 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6c44102ea0 0x7f6c44197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.517+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/1904851385 >> 172.21.15.100:0/1904851385 conn(0x7f6c440fd280 msgr2=0x7f6c441005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.517+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/1904851385 shutdown_connections 2024-02-14T17:42:09.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:09.517+0000 7f6c4ae5b700 1 -- 172.21.15.100:0/1904851385 wait complete. 2024-02-14T17:42:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:09 smithi179 ceph-mon[33633]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:09 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:09 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:09 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:09 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:09 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:09 smithi179 ceph-mon[33633]: from='osd.1 [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:42:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:09 smithi179 ceph-mon[33633]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-14T17:42:10.156 DEBUG:teuthology.orchestra.run.smithi100:osd.1> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.1.service 2024-02-14T17:42:10.158 INFO:tasks.cephadm:Deploying osd.2 on smithi100 with /dev/vg_nvme/lv_2... 2024-02-14T17:42:10.159 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-14T17:42:10.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[30138]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T17:42:10.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[30138]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T17:42:10.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:10.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[30138]: from='osd.1 [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:42:10.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[30138]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:42:10.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:10.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:10.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:10.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[30138]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:10.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[39557]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T17:42:10.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[39557]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T17:42:10.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:10.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[39557]: from='osd.1 [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:42:10.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[39557]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:42:10.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:10.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:10.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:10.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-mon[39557]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:10.402 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1[49905]: 2024-02-14T17:42:10.347+0000 7ff68fc52700 -1 osd.1 0 waiting for initial osdmap 2024-02-14T17:42:10.402 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:42:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1[49905]: 2024-02-14T17:42:10.351+0000 7ff68ba45700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:42:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:10 smithi179 ceph-mon[33633]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-14T17:42:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:10 smithi179 ceph-mon[33633]: osdmap e11: 2 total, 1 up, 2 in 2024-02-14T17:42:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:10 smithi179 ceph-mon[33633]: from='osd.1 [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:42:10.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:10 smithi179 ceph-mon[33633]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:42:10.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:10.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:10.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:10 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:10.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:10 smithi179 ceph-mon[33633]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:10.931 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:11.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:11 smithi100 ceph-mon[30138]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:42:11.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:11 smithi100 ceph-mon[30138]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T17:42:11.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:11 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:11.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:11 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:11.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:11 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:11.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:11 smithi100 ceph-mon[39557]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:42:11.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:11 smithi100 ceph-mon[39557]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T17:42:11.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:11.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:11.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:11 smithi179 ceph-mon[33633]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:42:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:11 smithi179 ceph-mon[33633]: osdmap e12: 2 total, 1 up, 2 in 2024-02-14T17:42:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[30138]: purged_snaps scrub starts 2024-02-14T17:42:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[30138]: purged_snaps scrub ok 2024-02-14T17:42:12.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[30138]: osd.1 [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537] boot 2024-02-14T17:42:12.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[30138]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T17:42:12.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:12.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[30138]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:12.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[39557]: purged_snaps scrub starts 2024-02-14T17:42:12.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[39557]: purged_snaps scrub ok 2024-02-14T17:42:12.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[39557]: osd.1 [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537] boot 2024-02-14T17:42:12.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[39557]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T17:42:12.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:12.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:12 smithi100 ceph-mon[39557]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:12 smithi179 ceph-mon[33633]: purged_snaps scrub starts 2024-02-14T17:42:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:12 smithi179 ceph-mon[33633]: purged_snaps scrub ok 2024-02-14T17:42:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:12 smithi179 ceph-mon[33633]: osd.1 [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537] boot 2024-02-14T17:42:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:12 smithi179 ceph-mon[33633]: osdmap e13: 2 total, 2 up, 2 in 2024-02-14T17:42:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:12 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:42:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:12 smithi179 ceph-mon[33633]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-14T17:42:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:13 smithi100 ceph-mon[30138]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T17:42:13.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:13 smithi100 ceph-mon[39557]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T17:42:13.677 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-14T17:42:13.677 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-14T17:42:13.677 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-02-14T17:42:13.677 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-02-14T17:42:13.677 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125642 s, 835 MB/s 2024-02-14T17:42:13.678 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-02-14T17:42:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:13 smithi179 ceph-mon[33633]: osdmap e14: 2 total, 2 up, 2 in 2024-02-14T17:42:14.461 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:14 smithi100 ceph-mon[30138]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:14.462 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:14 smithi100 ceph-mon[39557]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:14 smithi179 ceph-mon[33633]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:15.016 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-02-14T17:42:15.255 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:15.617 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[39557]: Detected new or changed devices on smithi100 2024-02-14T17:42:15.617 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.617 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.617 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi100 to 8032M 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[30138]: Detected new or changed devices on smithi100 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:15.618 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:15.619 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi100 to 8032M 2024-02-14T17:42:15.619 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.619 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:15.619 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:15.619 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:15 smithi179 ceph-mon[33633]: Detected new or changed devices on smithi100 2024-02-14T17:42:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:15 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi100 to 8032M 2024-02-14T17:42:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:16.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:16 smithi100 ceph-mon[30138]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:16 smithi100 ceph-mon[39557]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:16 smithi179 ceph-mon[33633]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:16.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 -- 172.21.15.100:0/1036430814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd164102740 msgr2=0x7fd164108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:16.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 --2- 172.21.15.100:0/1036430814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd164102740 0x7fd164108ff0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00b0a0 tx=0x7fd15c00b3b0 comp rx=0 tx=0).stop 2024-02-14T17:42:16.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 -- 172.21.15.100:0/1036430814 shutdown_connections 2024-02-14T17:42:16.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 --2- 172.21.15.100:0/1036430814 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd164102740 0x7fd164108ff0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:16.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 --2- 172.21.15.100:0/1036430814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd164100ba0 0x7fd164100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:16.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 --2- 172.21.15.100:0/1036430814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd164101da0 0x7fd164102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:16.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 -- 172.21.15.100:0/1036430814 >> 172.21.15.100:0/1036430814 conn(0x7fd1640fc140 msgr2=0x7fd1640fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:16.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 -- 172.21.15.100:0/1036430814 shutdown_connections 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 -- 172.21.15.100:0/1036430814 wait complete. 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 Processor -- start 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.903+0000 7fd169a08700 1 -- start start 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd169a08700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd164100ba0 0x7fd16410b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd169a08700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd164101da0 0x7fd164109b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd169a08700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd164102740 0x7fd16410a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd169a08700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16410a7c0 con 0x7fd164100ba0 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd169a08700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16410a930 con 0x7fd164101da0 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd169a08700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1641a13a0 con 0x7fd164102740 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd162ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd164100ba0 0x7fd16410b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd1637fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd164102740 0x7fd16410a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd1637fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd164102740 0x7fd16410a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:57392/0 (socket says 172.21.15.100:57392) 2024-02-14T17:42:16.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd162ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd164100ba0 0x7fd16410b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51996/0 (socket says 172.21.15.100:51996) 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd1637fe700 1 -- 172.21.15.100:0/3461096448 learned_addr learned my addr 172.21.15.100:0/3461096448 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd15a7fc700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd164101da0 0x7fd164109b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd1637fe700 1 -- 172.21.15.100:0/3461096448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd164101da0 msgr2=0x7fd164109b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd1637fe700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd164101da0 0x7fd164109b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd1637fe700 1 -- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd164100ba0 msgr2=0x7fd16410b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd1637fe700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd164100ba0 0x7fd16410b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd1637fe700 1 -- 172.21.15.100:0/3461096448 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd154009520 con 0x7fd164102740 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd162ffd700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd164100ba0 0x7fd16410b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.904+0000 7fd1637fe700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd164102740 0x7fd16410a040 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00b780 tx=0x7fd15c00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.905+0000 7fd168a06700 1 -- 172.21.15.100:0/3461096448 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00e040 con 0x7fd164102740 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.905+0000 7fd168a06700 1 -- 172.21.15.100:0/3461096448 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd15c004e60 con 0x7fd164102740 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.905+0000 7fd169a08700 1 -- 172.21.15.100:0/3461096448 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c009d00 con 0x7fd164102740 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.905+0000 7fd168a06700 1 -- 172.21.15.100:0/3461096448 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c005040 con 0x7fd164102740 2024-02-14T17:42:16.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.905+0000 7fd169a08700 1 -- 172.21.15.100:0/3461096448 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1641a1900 con 0x7fd164102740 2024-02-14T17:42:16.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.906+0000 7fd169a08700 1 -- 172.21.15.100:0/3461096448 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd164108080 con 0x7fd164102740 2024-02-14T17:42:16.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.906+0000 7fd168a06700 1 -- 172.21.15.100:0/3461096448 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 14) v1 ==== 91351+0+0 (secure 0 0 0) 0x7fd15c01d470 con 0x7fd164102740 2024-02-14T17:42:16.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.907+0000 7fd168a06700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd15006d940 0x7fd15006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:42:16.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.907+0000 7fd168a06700 1 -- 172.21.15.100:0/3461096448 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2199+0+0 (secure 0 0 0) 0x7fd15c08d7d0 con 0x7fd164102740 2024-02-14T17:42:16.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.907+0000 7fd162ffd700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd15006d940 0x7fd15006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:42:16.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.907+0000 7fd162ffd700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd15006d940 0x7fd15006fe00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd154009fd0 tx=0x7fd154009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:42:16.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:16.912+0000 7fd168a06700 1 -- 172.21.15.100:0/3461096448 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd15c0580e0 con 0x7fd164102740 2024-02-14T17:42:17.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:17.115+0000 7fd169a08700 1 -- 172.21.15.100:0/3461096448 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd16410cd70 con 0x7fd15006d940 2024-02-14T17:42:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:17 smithi179 ceph-mon[33633]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:42:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:42:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:17.830 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:17 smithi100 ceph-mon[30138]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:17.830 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:42:17.830 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:42:17.830 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:17.830 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:17 smithi100 ceph-mon[39557]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:17.830 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:42:17.830 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:42:17.831 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:18.664 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:18 smithi100 ceph-mon[30138]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:18.664 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:18 smithi100 ceph-mon[39557]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:18 smithi179 ceph-mon[33633]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1329708034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e476ae41-e2f9-43fa-8cca-7f766e005494"}]: dispatch 2024-02-14T17:42:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1329708034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e476ae41-e2f9-43fa-8cca-7f766e005494"}]': finished 2024-02-14T17:42:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:19 smithi100 ceph-mon[30138]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T17:42:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:19 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1329708034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e476ae41-e2f9-43fa-8cca-7f766e005494"}]: dispatch 2024-02-14T17:42:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1329708034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e476ae41-e2f9-43fa-8cca-7f766e005494"}]': finished 2024-02-14T17:42:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:19 smithi100 ceph-mon[39557]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T17:42:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:19 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1329708034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e476ae41-e2f9-43fa-8cca-7f766e005494"}]: dispatch 2024-02-14T17:42:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1329708034' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e476ae41-e2f9-43fa-8cca-7f766e005494"}]': finished 2024-02-14T17:42:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:19 smithi179 ceph-mon[33633]: osdmap e15: 3 total, 2 up, 3 in 2024-02-14T17:42:20.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:19 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:21.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:20 smithi100 ceph-mon[30138]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:21.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1978792372' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:42:21.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:20 smithi100 ceph-mon[39557]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:21.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1978792372' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:42:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:20 smithi179 ceph-mon[33633]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1978792372' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:42:23.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:23 smithi100 ceph-mon[30138]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:23.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:23 smithi100 ceph-mon[39557]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:23 smithi179 ceph-mon[33633]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:25.200 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:25 smithi100 ceph-mon[30138]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:25.200 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:25 smithi100 ceph-mon[39557]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:25 smithi179 ceph-mon[33633]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:27.140 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:27 smithi100 ceph-mon[39557]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:27.140 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:27 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T17:42:27.140 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:27 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:27.140 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:27 smithi100 ceph-mon[30138]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:27.140 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:27 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T17:42:27.141 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:27 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:27 smithi179 ceph-mon[33633]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:27.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:27 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-14T17:42:27.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:27 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:28.347 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:28 smithi100 ceph-mon[30138]: Deploying daemon osd.2 on smithi100 2024-02-14T17:42:28.348 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:28 smithi100 ceph-mon[39557]: Deploying daemon osd.2 on smithi100 2024-02-14T17:42:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:28 smithi179 ceph-mon[33633]: Deploying daemon osd.2 on smithi100 2024-02-14T17:42:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:29 smithi100 ceph-mon[30138]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:29.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:29 smithi100 ceph-mon[39557]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:29 smithi179 ceph-mon[33633]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:31 smithi100 ceph-mon[30138]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:31.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:31 smithi100 ceph-mon[39557]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:31 smithi179 ceph-mon[33633]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:32 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:32 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:32 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:32 smithi100 ceph-mon[30138]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:32 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:32 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:32 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:32 smithi100 ceph-mon[39557]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:32 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:32 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:32 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:32 smithi179 ceph-mon[33633]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:35 smithi100 ceph-mon[39557]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:35 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:35 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:35 smithi100 ceph-mon[39557]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:42:35.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:35 smithi100 ceph-mon[30138]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:35.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:35 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:35.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:35 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:35.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:35 smithi100 ceph-mon[30138]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:42:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:35 smithi179 ceph-mon[33633]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:35 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:35.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:35 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:35.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:35 smithi179 ceph-mon[33633]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-14T17:42:35.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.817+0000 7fd168a06700 1 -- 172.21.15.100:0/3461096448 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd16410cd70 con 0x7fd15006d940 2024-02-14T17:42:35.819 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 2 on host 'smithi100' 2024-02-14T17:42:35.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.819+0000 7fd169a08700 1 -- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd15006d940 msgr2=0x7fd15006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:35.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.819+0000 7fd169a08700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd15006d940 0x7fd15006fe00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd154009fd0 tx=0x7fd154009260 comp rx=0 tx=0).stop 2024-02-14T17:42:35.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.819+0000 7fd169a08700 1 -- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd164102740 msgr2=0x7fd16410a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:35.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.819+0000 7fd169a08700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd164102740 0x7fd16410a040 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00b780 tx=0x7fd15c00bce0 comp rx=0 tx=0).stop 2024-02-14T17:42:35.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.819+0000 7fd169a08700 1 -- 172.21.15.100:0/3461096448 shutdown_connections 2024-02-14T17:42:35.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.819+0000 7fd169a08700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd164102740 0x7fd16410a040 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:35.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.819+0000 7fd169a08700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd164100ba0 0x7fd16410b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:35.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.819+0000 7fd169a08700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fd15006d940 0x7fd15006fe00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:35.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.819+0000 7fd169a08700 1 --2- 172.21.15.100:0/3461096448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd164101da0 0x7fd164109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:35.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.819+0000 7fd169a08700 1 -- 172.21.15.100:0/3461096448 >> 172.21.15.100:0/3461096448 conn(0x7fd1640fc140 msgr2=0x7fd1641051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:35.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.820+0000 7fd169a08700 1 -- 172.21.15.100:0/3461096448 shutdown_connections 2024-02-14T17:42:35.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:35.820+0000 7fd169a08700 1 -- 172.21.15.100:0/3461096448 wait complete. 2024-02-14T17:42:36.437 DEBUG:teuthology.orchestra.run.smithi100:osd.2> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.2.service 2024-02-14T17:42:36.439 INFO:tasks.cephadm:Deploying osd.3 on smithi100 with /dev/vg_nvme/lv_1... 2024-02-14T17:42:36.439 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-14T17:42:37.012 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[39557]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[39557]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[39557]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[39557]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[30138]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[30138]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[30138]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:42:37.013 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:37.014 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:36 smithi100 ceph-mon[30138]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:37.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:36 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:37.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:36 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:37.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:36 smithi179 ceph-mon[33633]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-14T17:42:37.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:36 smithi179 ceph-mon[33633]: osdmap e16: 3 total, 2 up, 3 in 2024-02-14T17:42:37.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:36 smithi179 ceph-mon[33633]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:42:37.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:36 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:37.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:36 smithi179 ceph-mon[33633]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:37.262 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:37.348 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 17:42:37 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2[56161]: 2024-02-14T17:42:37.015+0000 7f0a53302700 -1 osd.2 0 waiting for initial osdmap 2024-02-14T17:42:37.349 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 17:42:37 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2[56161]: 2024-02-14T17:42:37.018+0000 7f0a4f0f5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:42:38.179 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[30138]: purged_snaps scrub starts 2024-02-14T17:42:38.179 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[30138]: purged_snaps scrub ok 2024-02-14T17:42:38.179 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[30138]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:42:38.179 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[30138]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T17:42:38.180 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:38.180 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:38.180 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[39557]: purged_snaps scrub starts 2024-02-14T17:42:38.180 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[39557]: purged_snaps scrub ok 2024-02-14T17:42:38.180 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[39557]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:42:38.180 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[39557]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T17:42:38.180 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:38.180 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:38 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:38.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:38 smithi179 ceph-mon[33633]: purged_snaps scrub starts 2024-02-14T17:42:38.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:38 smithi179 ceph-mon[33633]: purged_snaps scrub ok 2024-02-14T17:42:38.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:38 smithi179 ceph-mon[33633]: from='osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:42:38.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:38 smithi179 ceph-mon[33633]: osdmap e17: 3 total, 2 up, 3 in 2024-02-14T17:42:38.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:38 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:38.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:38 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:39.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:39.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[39557]: osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896] boot 2024-02-14T17:42:39.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[39557]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T17:42:39.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:39.209 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[39557]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:39.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:42:39.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:39.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[30138]: osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896] boot 2024-02-14T17:42:39.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[30138]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T17:42:39.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:39.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[30138]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:39.210 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:39 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:42:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:39 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:39 smithi179 ceph-mon[33633]: osd.2 [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896] boot 2024-02-14T17:42:39.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:39 smithi179 ceph-mon[33633]: osdmap e18: 3 total, 3 up, 3 in 2024-02-14T17:42:39.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:39 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:42:39.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:39 smithi179 ceph-mon[33633]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-14T17:42:39.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:39 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:42:40.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T17:42:40.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T17:42:40.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:42:40.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:40.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:40.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:40.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:40.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:40.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:40.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:40.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:40 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:40.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: osdmap e19: 3 total, 3 up, 3 in 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:40.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:40 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:40.705 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60381]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:42:40.705 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60381]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:40.705 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60381]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:40.705 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60381]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:40.759 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-14T17:42:40.760 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-14T17:42:40.760 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-02-14T17:42:40.760 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-02-14T17:42:40.760 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148942 s, 704 MB/s 2024-02-14T17:42:40.760 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-02-14T17:42:40.978 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60405]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:42:40.978 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60405]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:40.978 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60405]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:40.978 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60405]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:41.240 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[39557]: Detected new or changed devices on smithi100 2024-02-14T17:42:41.240 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi100 to 5354M 2024-02-14T17:42:41.240 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:41.240 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:42:41.241 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[39557]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T17:42:41.241 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[39557]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:41.241 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:41.241 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:41.241 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60505]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:42:41.241 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:41.241 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60505]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:41.241 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:42:40 smithi100 sudo[60505]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:41.241 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[30138]: Detected new or changed devices on smithi100 2024-02-14T17:42:41.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi100 to 5354M 2024-02-14T17:42:41.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:41.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:42:41.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[30138]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T17:42:41.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[30138]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:41.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:41.242 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:41 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:41.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:41 smithi179 ceph-mon[33633]: Detected new or changed devices on smithi100 2024-02-14T17:42:41.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:41 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi100 to 5354M 2024-02-14T17:42:41.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:41 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:42:41.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:41 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:42:41.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:41 smithi179 ceph-mon[33633]: osdmap e20: 3 total, 3 up, 3 in 2024-02-14T17:42:41.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:41 smithi179 ceph-mon[33633]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:41.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:41 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:41.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:41 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:41.394 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-02-14T17:42:41.495 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60516]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:42:41.495 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60516]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:41.495 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60516]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:41.495 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60516]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:41.636 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:42:41.758 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60538]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:42:41.806 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60538]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:41.806 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60538]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:41.806 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60538]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:42.012 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60597]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:42:42.012 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60597]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:42.012 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60597]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:42.013 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 17:42:41 smithi100 sudo[60597]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:42.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 sudo[60619]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:42:42.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 sudo[60619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:42.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 sudo[60619]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:42.272 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 sudo[60619]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:42.534 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 ceph-mon[33633]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T17:42:42.534 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 ceph-mon[33633]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-14T17:42:42.534 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:42:42.534 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 sudo[36449]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:42:42.534 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 sudo[36449]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:42.534 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 sudo[36449]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:42.534 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 sudo[36449]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:42.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:42 smithi100 ceph-mon[39557]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T17:42:42.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:42 smithi100 ceph-mon[39557]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-14T17:42:42.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:42 smithi100 ceph-mon[39557]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:42:42.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 sudo[60643]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:42:42.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 sudo[60643]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:42.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 sudo[60643]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:42.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 sudo[60643]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:42.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 ceph-mon[30138]: osdmap e21: 3 total, 3 up, 3 in 2024-02-14T17:42:42.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 ceph-mon[30138]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-14T17:42:42.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:42 smithi100 ceph-mon[30138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:42:42.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 sudo[36452]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:42:42.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 sudo[36452]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:42.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 sudo[36452]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:42.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:42 smithi179 sudo[36452]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:43.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 sudo[60672]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-14T17:42:43.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 sudo[60672]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:43.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 sudo[60672]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:43.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 sudo[60672]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.327+0000 7fbbbc792700 1 -- 172.21.15.100:0/2215776692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4102740 msgr2=0x7fbbb4108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.327+0000 7fbbbc792700 1 --2- 172.21.15.100:0/2215776692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4102740 0x7fbbb4108ff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009fc0 tx=0x7fbbac00b330 comp rx=0 tx=0).stop 2024-02-14T17:42:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.327+0000 7fbbbc792700 1 -- 172.21.15.100:0/2215776692 shutdown_connections 2024-02-14T17:42:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.327+0000 7fbbbc792700 1 --2- 172.21.15.100:0/2215776692 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4102740 0x7fbbb4108ff0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.327+0000 7fbbbc792700 1 --2- 172.21.15.100:0/2215776692 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbb4100ba0 0x7fbbb4100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.327+0000 7fbbbc792700 1 --2- 172.21.15.100:0/2215776692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb4101da0 0x7fbbb4102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.327+0000 7fbbbc792700 1 -- 172.21.15.100:0/2215776692 >> 172.21.15.100:0/2215776692 conn(0x7fbbb40fc140 msgr2=0x7fbbb40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:42:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.327+0000 7fbbbc792700 1 -- 172.21.15.100:0/2215776692 shutdown_connections 2024-02-14T17:42:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.327+0000 7fbbbc792700 1 -- 172.21.15.100:0/2215776692 wait complete. 2024-02-14T17:42:43.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbbc792700 1 Processor -- start 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbbc792700 1 -- start start 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbbc792700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbb4100ba0 0x7fbbb4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbbc792700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4101da0 0x7fbbb41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbbc792700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb4102740 0x7fbbb419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbbc792700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb419ee20 con 0x7fbbb4101da0 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbbc792700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb419ef60 con 0x7fbbb4102740 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbbc792700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb419f0f0 con 0x7fbbb4100ba0 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbba52e700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbb4100ba0 0x7fbbb4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbb9d2d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4101da0 0x7fbbb41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbba52e700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbb4100ba0 0x7fbbb4197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:43220/0 (socket says 172.21.15.100:43220) 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbba52e700 1 -- 172.21.15.100:0/3682615954 learned_addr learned my addr 172.21.15.100:0/3682615954 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:42:43.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbbad2f700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb4102740 0x7fbbb419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbba52e700 1 -- 172.21.15.100:0/3682615954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb4102740 msgr2=0x7fbbb419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbba52e700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb4102740 0x7fbbb419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbba52e700 1 -- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4101da0 msgr2=0x7fbbb41977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbba52e700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4101da0 0x7fbbb41977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbba52e700 1 -- 172.21.15.100:0/3682615954 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00095f0 con 0x7fbbb4100ba0 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.328+0000 7fbbbad2f700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb4102740 0x7fbbb419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.329+0000 7fbbba52e700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbb4100ba0 0x7fbbb4197290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ec20 tx=0x7fbbb000ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.329+0000 7fbbab7fe700 1 -- 172.21.15.100:0/3682615954 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0009a10 con 0x7fbbb4100ba0 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.329+0000 7fbbab7fe700 1 -- 172.21.15.100:0/3682615954 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fbbb000cd40 con 0x7fbbb4100ba0 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.329+0000 7fbbbc792700 1 -- 172.21.15.100:0/3682615954 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac009c40 con 0x7fbbb4100ba0 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.329+0000 7fbbab7fe700 1 -- 172.21.15.100:0/3682615954 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb00186b0 con 0x7fbbb4100ba0 2024-02-14T17:42:43.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.329+0000 7fbbbc792700 1 -- 172.21.15.100:0/3682615954 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb419f740 con 0x7fbbb4100ba0 2024-02-14T17:42:43.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.330+0000 7fbbbc792700 1 -- 172.21.15.100:0/3682615954 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb4108980 con 0x7fbbb4100ba0 2024-02-14T17:42:43.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.330+0000 7fbbab7fe700 1 -- 172.21.15.100:0/3682615954 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7fbbb0010430 con 0x7fbbb4100ba0 2024-02-14T17:42:43.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.331+0000 7fbbab7fe700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fbba006d9d0 0x7fbba006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:42:43.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.331+0000 7fbbb9d2d700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fbba006d9d0 0x7fbba006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:42:43.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.331+0000 7fbbab7fe700 1 -- 172.21.15.100:0/3682615954 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3017+0+0 (secure 0 0 0) 0x7fbbb0014070 con 0x7fbbb4100ba0 2024-02-14T17:42:43.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.331+0000 7fbbb9d2d700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fbba006d9d0 0x7fbba006fe90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbba4007be0 tx=0x7fbba40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:42:43.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.335+0000 7fbbab7fe700 1 -- 172.21.15.100:0/3682615954 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbb00571b0 con 0x7fbbb4100ba0 2024-02-14T17:42:43.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:42:43.538+0000 7fbbbc792700 1 -- 172.21.15.100:0/3682615954 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbb410bc80 con 0x7fbba006d9d0 2024-02-14T17:42:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:42:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:42:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:42:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:42:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:43.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:43.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:43.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:43.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 sudo[60702]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-14T17:42:43.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 sudo[60702]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-14T17:42:43.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 sudo[60702]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-14T17:42:43.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 sudo[60702]: pam_unix(sudo:session): session closed for user root 2024-02-14T17:42:43.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:43.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:42:43.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:43.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:43.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:43.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:42:43.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:43.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:43.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:43.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:42:43.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:43.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:42:43.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:42:43.656 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:42:44.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[30138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[30138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[30138]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[30138]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[39557]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[39557]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[39557]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:42:44.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:42:44.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:44.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:44 smithi100 ceph-mon[39557]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:44 smithi179 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-14T17:42:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:44 smithi179 ceph-mon[33633]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-14T17:42:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:44 smithi179 ceph-mon[33633]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:42:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:42:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:42:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:44 smithi179 ceph-mon[33633]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[30138]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/450957232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a354e22-5aff-4f94-8764-0b3dfbca48f7"}]: dispatch 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/450957232' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a354e22-5aff-4f94-8764-0b3dfbca48f7"}]': finished 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[30138]: osdmap e22: 4 total, 3 up, 4 in 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4039944307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[39557]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/450957232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a354e22-5aff-4f94-8764-0b3dfbca48f7"}]: dispatch 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/450957232' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a354e22-5aff-4f94-8764-0b3dfbca48f7"}]': finished 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[39557]: osdmap e22: 4 total, 3 up, 4 in 2024-02-14T17:42:47.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:42:47.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4039944307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:42:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:47 smithi179 ceph-mon[33633]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:42:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/450957232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a354e22-5aff-4f94-8764-0b3dfbca48f7"}]: dispatch 2024-02-14T17:42:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/450957232' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a354e22-5aff-4f94-8764-0b3dfbca48f7"}]': finished 2024-02-14T17:42:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:47 smithi179 ceph-mon[33633]: osdmap e22: 4 total, 3 up, 4 in 2024-02-14T17:42:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:42:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4039944307' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:42:49.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:49 smithi100 ceph-mon[30138]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-02-14T17:42:49.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:49 smithi100 ceph-mon[39557]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-02-14T17:42:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:49 smithi179 ceph-mon[33633]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-02-14T17:42:51.396 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:51 smithi100 ceph-mon[30138]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-14T17:42:51.397 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:51 smithi100 ceph-mon[39557]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-14T17:42:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:51 smithi179 ceph-mon[33633]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-14T17:42:53.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:53 smithi100 ceph-mon[30138]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-14T17:42:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:53 smithi100 ceph-mon[39557]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-14T17:42:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:53 smithi179 ceph-mon[33633]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-02-14T17:42:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-14T17:42:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:54 smithi100 ceph-mon[30138]: Deploying daemon osd.3 on smithi100 2024-02-14T17:42:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-14T17:42:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:54 smithi100 ceph-mon[39557]: Deploying daemon osd.3 on smithi100 2024-02-14T17:42:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-14T17:42:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:42:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:54 smithi179 ceph-mon[33633]: Deploying daemon osd.3 on smithi100 2024-02-14T17:42:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:55 smithi100 ceph-mon[30138]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:42:55.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:55 smithi100 ceph-mon[39557]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:42:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:55 smithi179 ceph-mon[33633]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:42:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:57 smithi100 ceph-mon[30138]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:42:57.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:57 smithi100 ceph-mon[39557]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:42:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:57 smithi179 ceph-mon[33633]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-02-14T17:42:59.396 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:59 smithi100 ceph-mon[39557]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-02-14T17:42:59.396 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:59 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:59.396 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:59 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:59.396 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:42:59 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:59.397 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:59 smithi100 ceph-mon[30138]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-02-14T17:42:59.397 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:59.397 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:59.397 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:42:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:42:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:59 smithi179 ceph-mon[33633]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 67 KiB/s wr, 4 op/s 2024-02-14T17:42:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:42:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:42:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:01.248 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:01 smithi100 ceph-mon[30138]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:43:01.248 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:01 smithi100 ceph-mon[39557]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:43:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:01 smithi179 ceph-mon[33633]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:43:02.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:02 smithi100 ceph-mon[30138]: from='osd.3 [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T17:43:02.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:02 smithi100 ceph-mon[30138]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T17:43:02.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:02 smithi100 ceph-mon[39557]: from='osd.3 [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T17:43:02.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:02 smithi100 ceph-mon[39557]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T17:43:02.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:02 smithi179 ceph-mon[33633]: from='osd.3 [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T17:43:02.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:02 smithi179 ceph-mon[33633]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-14T17:43:02.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.592+0000 7fbbab7fe700 1 -- 172.21.15.100:0/3682615954 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbbb410bc80 con 0x7fbba006d9d0 2024-02-14T17:43:02.594 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 3 on host 'smithi100' 2024-02-14T17:43:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 -- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fbba006d9d0 msgr2=0x7fbba006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fbba006d9d0 0x7fbba006fe90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbba4007be0 tx=0x7fbba40073d0 comp rx=0 tx=0).stop 2024-02-14T17:43:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 -- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbb4100ba0 msgr2=0x7fbbb4197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:02.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbb4100ba0 0x7fbbb4197290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000ec20 tx=0x7fbbb000ef30 comp rx=0 tx=0).stop 2024-02-14T17:43:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 -- 172.21.15.100:0/3682615954 shutdown_connections 2024-02-14T17:43:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fbba006d9d0 0x7fbba006fe90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb4102740 0x7fbbb419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbb4100ba0 0x7fbbb4197290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 --2- 172.21.15.100:0/3682615954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4101da0 0x7fbbb41977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 -- 172.21.15.100:0/3682615954 >> 172.21.15.100:0/3682615954 conn(0x7fbbb40fc140 msgr2=0x7fbbb406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 -- 172.21.15.100:0/3682615954 shutdown_connections 2024-02-14T17:43:02.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:43:02.595+0000 7fbbbc792700 1 -- 172.21.15.100:0/3682615954 wait complete. 2024-02-14T17:43:03.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[30138]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:43:03.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[30138]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-14T17:43:03.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[30138]: osdmap e23: 4 total, 3 up, 4 in 2024-02-14T17:43:03.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[30138]: from='osd.3 [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[30138]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[39557]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[39557]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[39557]: osdmap e23: 4 total, 3 up, 4 in 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[39557]: from='osd.3 [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[39557]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:03 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:03.212 DEBUG:teuthology.orchestra.run.smithi100:osd.3> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.3.service 2024-02-14T17:43:03.214 INFO:tasks.cephadm:Deploying osd.4 on smithi179 with /dev/vg_nvme/lv_4... 2024-02-14T17:43:03.215 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-14T17:43:03.461 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:43:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:03 smithi179 ceph-mon[33633]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:43:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:03 smithi179 ceph-mon[33633]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-14T17:43:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:03 smithi179 ceph-mon[33633]: osdmap e23: 4 total, 3 up, 4 in 2024-02-14T17:43:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:03 smithi179 ceph-mon[33633]: from='osd.3 [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:43:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:03 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:03 smithi179 ceph-mon[33633]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-02-14T17:43:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:03 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:03 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:04.386 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[30138]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:43:04.386 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[30138]: osdmap e24: 4 total, 3 up, 4 in 2024-02-14T17:43:04.386 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:04.386 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:04.387 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:04.387 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:04.387 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[39557]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:43:04.387 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[39557]: osdmap e24: 4 total, 3 up, 4 in 2024-02-14T17:43:04.387 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:04.387 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:04.387 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:04.387 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:04 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:04.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:04 smithi179 ceph-mon[33633]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-02-14T17:43:04.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:04 smithi179 ceph-mon[33633]: osdmap e24: 4 total, 3 up, 4 in 2024-02-14T17:43:04.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:04 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:04.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:04 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:04.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:04 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:04.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:04 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:05.277 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-14T17:43:05.277 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-14T17:43:05.277 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-02-14T17:43:05.277 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-02-14T17:43:05.277 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011415 s, 919 MB/s 2024-02-14T17:43:05.277 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-02-14T17:43:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[30138]: purged_snaps scrub starts 2024-02-14T17:43:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[30138]: purged_snaps scrub ok 2024-02-14T17:43:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[30138]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[30138]: osd.3 [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967] boot 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[30138]: osdmap e25: 4 total, 4 up, 4 in 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[39557]: purged_snaps scrub starts 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[39557]: purged_snaps scrub ok 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[39557]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[39557]: osd.3 [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967] boot 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[39557]: osdmap e25: 4 total, 4 up, 4 in 2024-02-14T17:43:05.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:05 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:05 smithi179 ceph-mon[33633]: purged_snaps scrub starts 2024-02-14T17:43:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:05 smithi179 ceph-mon[33633]: purged_snaps scrub ok 2024-02-14T17:43:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:05 smithi179 ceph-mon[33633]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-14T17:43:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:05 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:05.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:05 smithi179 ceph-mon[33633]: osd.3 [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967] boot 2024-02-14T17:43:05.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:05 smithi179 ceph-mon[33633]: osdmap e25: 4 total, 4 up, 4 in 2024-02-14T17:43:05.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:05 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:43:05.896 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi179:vg_nvme/lv_4 2024-02-14T17:43:06.133 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:43:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:06 smithi179 ceph-mon[33633]: osdmap e26: 4 total, 4 up, 4 in 2024-02-14T17:43:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:06 smithi179 ceph-mon[33633]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:06.897 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:06 smithi100 ceph-mon[39557]: osdmap e26: 4 total, 4 up, 4 in 2024-02-14T17:43:06.897 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:06 smithi100 ceph-mon[39557]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:06.897 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:06 smithi100 ceph-mon[30138]: osdmap e26: 4 total, 4 up, 4 in 2024-02-14T17:43:06.898 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:06 smithi100 ceph-mon[30138]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.755+0000 7f1599e04700 1 -- 172.21.15.179:0/1052337339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15941079c0 msgr2=0x7f1594109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.755+0000 7f1599e04700 1 --2- 172.21.15.179:0/1052337339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15941079c0 0x7f1594109e30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f158c00b0a0 tx=0x7f158c00b3b0 comp rx=0 tx=0).stop 2024-02-14T17:43:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.755+0000 7f1599e04700 1 -- 172.21.15.179:0/1052337339 shutdown_connections 2024-02-14T17:43:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.755+0000 7f1599e04700 1 --2- 172.21.15.179:0/1052337339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15941079c0 0x7f1594109e30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.755+0000 7f1599e04700 1 --2- 172.21.15.179:0/1052337339 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1594102760 0x7f1594104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.755+0000 7f1599e04700 1 --2- 172.21.15.179:0/1052337339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1594105090 0x7f1594107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.755+0000 7f1599e04700 1 -- 172.21.15.179:0/1052337339 >> 172.21.15.179:0/1052337339 conn(0x7f15940fc170 msgr2=0x7f15940fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.755+0000 7f1599e04700 1 -- 172.21.15.179:0/1052337339 shutdown_connections 2024-02-14T17:43:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.755+0000 7f1599e04700 1 -- 172.21.15.179:0/1052337339 wait complete. 2024-02-14T17:43:07.759 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.755+0000 7f1599e04700 1 Processor -- start 2024-02-14T17:43:07.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1599e04700 1 -- start start 2024-02-14T17:43:07.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1599e04700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1594102760 0x7f1594197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:07.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1599e04700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1594105090 0x7f1594197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:07.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1599e04700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15941079c0 0x7f159419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:07.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1599e04700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159419ede0 con 0x7f1594105090 2024-02-14T17:43:07.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1599e04700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159419ef20 con 0x7f15941079c0 2024-02-14T17:43:07.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1599e04700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159419f0b0 con 0x7f1594102760 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f15937fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1594102760 0x7f1594197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1592ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1594105090 0x7f1594197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1593fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15941079c0 0x7f159419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f15937fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1594102760 0x7f1594197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.179:56326/0 (socket says 172.21.15.179:56326) 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f15937fe700 1 -- 172.21.15.179:0/3348629188 learned_addr learned my addr 172.21.15.179:0/3348629188 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1593fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15941079c0 0x7f159419e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42498/0 (socket says 172.21.15.179:42498) 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1592ffd700 1 -- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1594102760 msgr2=0x7f1594197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1592ffd700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1594102760 0x7f1594197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1592ffd700 1 -- 172.21.15.179:0/3348629188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15941079c0 msgr2=0x7f159419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1592ffd700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15941079c0 0x7f159419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1592ffd700 1 -- 172.21.15.179:0/3348629188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c009d00 con 0x7f1594105090 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1593fff700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15941079c0 0x7f159419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:43:07.761 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f15937fe700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1594102760 0x7f1594197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:43:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.756+0000 7f1592ffd700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1594105090 0x7f1594197790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f158800d780 tx=0x7f158800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:43:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.757+0000 7f1590ff9700 1 -- 172.21.15.179:0/3348629188 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588004020 con 0x7f1594105090 2024-02-14T17:43:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.757+0000 7f1590ff9700 1 -- 172.21.15.179:0/3348629188 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1588004830 con 0x7f1594105090 2024-02-14T17:43:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.757+0000 7f1590ff9700 1 -- 172.21.15.179:0/3348629188 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588005410 con 0x7f1594105090 2024-02-14T17:43:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.757+0000 7f1599e04700 1 -- 172.21.15.179:0/3348629188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159419f390 con 0x7f1594105090 2024-02-14T17:43:07.762 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.757+0000 7f1599e04700 1 -- 172.21.15.179:0/3348629188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159419f840 con 0x7f1594105090 2024-02-14T17:43:07.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.758+0000 7f1599e04700 1 -- 172.21.15.179:0/3348629188 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15940fdcf0 con 0x7f1594105090 2024-02-14T17:43:07.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.758+0000 7f1590ff9700 1 -- 172.21.15.179:0/3348629188 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7f1588004c80 con 0x7f1594105090 2024-02-14T17:43:07.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.759+0000 7f1590ff9700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f157c06d9d0 0x7f157c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:07.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.759+0000 7f1590ff9700 1 -- 172.21.15.179:0/3348629188 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3449+0+0 (secure 0 0 0) 0x7f158808c330 con 0x7f1594105090 2024-02-14T17:43:07.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.759+0000 7f15937fe700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f157c06d9d0 0x7f157c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:07.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.759+0000 7f15937fe700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f157c06d9d0 0x7f157c06fe90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f15840104e0 tx=0x7f1584007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:43:07.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.764+0000 7f1590ff9700 1 -- 172.21.15.179:0/3348629188 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1588056700 con 0x7f1594105090 2024-02-14T17:43:07.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:07.971+0000 7f1599e04700 1 -- 172.21.15.179:0/3348629188 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f15940620b0 con 0x7f157c06d9d0 2024-02-14T17:43:08.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: Detected new or changed devices on smithi100 2024-02-14T17:43:08.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.312 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi100 to 4016M 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:08.313 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: Detected new or changed devices on smithi100 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi100 to 4016M 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:08.314 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:08.315 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.315 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:08.315 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:08.315 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:08.315 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: Detected new or changed devices on smithi100 2024-02-14T17:43:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi100 to 4016M 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:09.222 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:08 smithi179 ceph-mon[33633]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:43:09.222 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:43:09.223 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:43:09.223 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:09.223 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:08 smithi179 ceph-mon[33633]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:09.237 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:08 smithi100 ceph-mon[39557]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:43:09.238 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:43:09.238 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:43:09.238 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:09.238 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:08 smithi100 ceph-mon[39557]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:09.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:08 smithi100 ceph-mon[30138]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:43:09.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:43:09.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:43:09.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:09.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:08 smithi100 ceph-mon[30138]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:11 smithi179 ceph-mon[33633]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/17765907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a753e42-d9e5-487a-b716-5ff9bd63af60"}]: dispatch 2024-02-14T17:43:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a753e42-d9e5-487a-b716-5ff9bd63af60"}]: dispatch 2024-02-14T17:43:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a753e42-d9e5-487a-b716-5ff9bd63af60"}]': finished 2024-02-14T17:43:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:11 smithi179 ceph-mon[33633]: osdmap e27: 5 total, 4 up, 5 in 2024-02-14T17:43:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:11.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[39557]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:11.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/17765907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a753e42-d9e5-487a-b716-5ff9bd63af60"}]: dispatch 2024-02-14T17:43:11.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a753e42-d9e5-487a-b716-5ff9bd63af60"}]: dispatch 2024-02-14T17:43:11.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a753e42-d9e5-487a-b716-5ff9bd63af60"}]': finished 2024-02-14T17:43:11.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[39557]: osdmap e27: 5 total, 4 up, 5 in 2024-02-14T17:43:11.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:11.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[30138]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:11.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/17765907' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a753e42-d9e5-487a-b716-5ff9bd63af60"}]: dispatch 2024-02-14T17:43:11.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a753e42-d9e5-487a-b716-5ff9bd63af60"}]: dispatch 2024-02-14T17:43:11.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a753e42-d9e5-487a-b716-5ff9bd63af60"}]': finished 2024-02-14T17:43:11.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[30138]: osdmap e27: 5 total, 4 up, 5 in 2024-02-14T17:43:11.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:11 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:12.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/1353821867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:43:12.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/1353821867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:43:12.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/1353821867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:43:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:13 smithi100 ceph-mon[39557]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:13 smithi100 ceph-mon[30138]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:13.442 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:13 smithi179 ceph-mon[33633]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:15.312 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:15 smithi179 ceph-mon[33633]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:15 smithi100 ceph-mon[30138]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:15.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:15 smithi100 ceph-mon[39557]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:17 smithi179 ceph-mon[33633]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-14T17:43:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:17 smithi100 ceph-mon[30138]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-14T17:43:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:17.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:17 smithi100 ceph-mon[39557]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:17.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-14T17:43:17.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:18 smithi100 ceph-mon[39557]: Deploying daemon osd.4 on smithi179 2024-02-14T17:43:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:18 smithi100 ceph-mon[30138]: Deploying daemon osd.4 on smithi179 2024-02-14T17:43:18.440 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:18 smithi179 ceph-mon[33633]: Deploying daemon osd.4 on smithi179 2024-02-14T17:43:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:19 smithi179 ceph-mon[33633]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:19.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:19 smithi100 ceph-mon[39557]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:19.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:19 smithi100 ceph-mon[30138]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:21 smithi100 ceph-mon[30138]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:21 smithi100 ceph-mon[39557]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:21.454 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:21 smithi179 ceph-mon[33633]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:23.261 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:23 smithi179 ceph-mon[33633]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:23.262 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:23 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:23.262 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:23 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:23.262 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:23 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:23.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:23 smithi100 ceph-mon[30138]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:23.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:23.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:23.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:23 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:23.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:23 smithi100 ceph-mon[39557]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:23.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:23 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:23.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:23 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:23.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:23 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:25 smithi179 ceph-mon[33633]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:25 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:25 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:25.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:25 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:25.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:25 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:25.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:25 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:25.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[30138]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:25.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:25.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:25.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:25.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:25.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:25.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[39557]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:25.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:25.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:25.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:25.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:25.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:25 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:25.984 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 4 on host 'smithi179' 2024-02-14T17:43:25.984 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.982+0000 7f1590ff9700 1 -- 172.21.15.179:0/3348629188 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f15940620b0 con 0x7f157c06d9d0 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 -- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f157c06d9d0 msgr2=0x7f157c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f157c06d9d0 0x7f157c06fe90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f15840104e0 tx=0x7f1584007480 comp rx=0 tx=0).stop 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 -- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1594105090 msgr2=0x7f1594197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1594105090 0x7f1594197790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f158800d780 tx=0x7f158800da90 comp rx=0 tx=0).stop 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 -- 172.21.15.179:0/3348629188 shutdown_connections 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f157c06d9d0 0x7f157c06fe90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15941079c0 0x7f159419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1594102760 0x7f1594197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 --2- 172.21.15.179:0/3348629188 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1594105090 0x7f1594197790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 -- 172.21.15.179:0/3348629188 >> 172.21.15.179:0/3348629188 conn(0x7f15940fc170 msgr2=0x7f15941033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 -- 172.21.15.179:0/3348629188 shutdown_connections 2024-02-14T17:43:25.987 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:25.985+0000 7f1599e04700 1 -- 172.21.15.179:0/3348629188 wait complete. 2024-02-14T17:43:26.296 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:26 smithi179 ceph-mon[33633]: from='osd.4 [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:43:26.296 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:26 smithi179 ceph-mon[33633]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:43:26.296 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:26 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:26.297 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:26 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:26.297 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:26 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:26.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:26 smithi100 ceph-mon[30138]: from='osd.4 [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:43:26.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:26 smithi100 ceph-mon[30138]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:43:26.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:26 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:26.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:26 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:26.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:26 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:26.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:26 smithi100 ceph-mon[39557]: from='osd.4 [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:43:26.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:26 smithi100 ceph-mon[39557]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-14T17:43:26.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:26 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:26.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:26 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:26.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:26 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:26.572 DEBUG:teuthology.orchestra.run.smithi179:osd.4> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.4.service 2024-02-14T17:43:26.575 INFO:tasks.cephadm:Deploying osd.5 on smithi179 with /dev/vg_nvme/lv_3... 2024-02-14T17:43:26.575 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-14T17:43:27.335 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:43:27.457 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:27 smithi179 ceph-mon[33633]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:27.458 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:27 smithi179 ceph-mon[33633]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-14T17:43:27.458 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:27 smithi179 ceph-mon[33633]: osdmap e28: 5 total, 4 up, 5 in 2024-02-14T17:43:27.458 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:27 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:27.458 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:27 smithi179 ceph-mon[33633]: from='osd.4 [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:27.458 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:27 smithi179 ceph-mon[33633]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[30138]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[30138]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-14T17:43:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[30138]: osdmap e28: 5 total, 4 up, 5 in 2024-02-14T17:43:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[30138]: from='osd.4 [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[30138]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[39557]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[39557]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-14T17:43:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[39557]: osdmap e28: 5 total, 4 up, 5 in 2024-02-14T17:43:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[39557]: from='osd.4 [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:27 smithi100 ceph-mon[39557]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:27.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 17:43:27 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T17:43:27.555+0000 7f078217d700 -1 osd.4 0 waiting for initial osdmap 2024-02-14T17:43:27.822 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 17:43:27 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T17:43:27.560+0000 7f0779f68700 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:43:28.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:28 smithi179 ceph-mon[33633]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:43:28.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:28 smithi179 ceph-mon[33633]: osdmap e29: 5 total, 4 up, 5 in 2024-02-14T17:43:28.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:28 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:28 smithi179 ceph-mon[33633]: from='osd.4 ' entity='osd.4' 2024-02-14T17:43:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:28 smithi179 ceph-mon[33633]: osd.4 [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533] boot 2024-02-14T17:43:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:28 smithi179 ceph-mon[33633]: osdmap e30: 5 total, 5 up, 5 in 2024-02-14T17:43:28.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:28 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[30138]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:43:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[30138]: osdmap e29: 5 total, 4 up, 5 in 2024-02-14T17:43:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[30138]: from='osd.4 ' entity='osd.4' 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[30138]: osd.4 [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533] boot 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[30138]: osdmap e30: 5 total, 5 up, 5 in 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[39557]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[39557]: osdmap e29: 5 total, 4 up, 5 in 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[39557]: from='osd.4 ' entity='osd.4' 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[39557]: osd.4 [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533] boot 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[39557]: osdmap e30: 5 total, 5 up, 5 in 2024-02-14T17:43:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:28 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:43:29.307 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:29 smithi179 ceph-mon[33633]: purged_snaps scrub starts 2024-02-14T17:43:29.307 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:29 smithi179 ceph-mon[33633]: purged_snaps scrub ok 2024-02-14T17:43:29.307 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:29 smithi179 ceph-mon[33633]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:29.307 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:29 smithi179 ceph-mon[33633]: osdmap e31: 5 total, 5 up, 5 in 2024-02-14T17:43:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:29 smithi100 ceph-mon[30138]: purged_snaps scrub starts 2024-02-14T17:43:29.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:29 smithi100 ceph-mon[30138]: purged_snaps scrub ok 2024-02-14T17:43:29.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:29 smithi100 ceph-mon[30138]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:29.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:29 smithi100 ceph-mon[30138]: osdmap e31: 5 total, 5 up, 5 in 2024-02-14T17:43:29.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:29 smithi100 ceph-mon[39557]: purged_snaps scrub starts 2024-02-14T17:43:29.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:29 smithi100 ceph-mon[39557]: purged_snaps scrub ok 2024-02-14T17:43:29.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:29 smithi100 ceph-mon[39557]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-14T17:43:29.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:29 smithi100 ceph-mon[39557]: osdmap e31: 5 total, 5 up, 5 in 2024-02-14T17:43:29.901 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-14T17:43:29.901 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-14T17:43:29.901 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-02-14T17:43:29.901 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-02-14T17:43:29.901 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121551 s, 863 MB/s 2024-02-14T17:43:29.901 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-02-14T17:43:31.248 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi179:vg_nvme/lv_3 2024-02-14T17:43:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[30138]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 499 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[30138]: osdmap e32: 5 total, 5 up, 5 in 2024-02-14T17:43:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[39557]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 499 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:31.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[39557]: osdmap e32: 5 total, 5 up, 5 in 2024-02-14T17:43:31.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:31.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:31.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:31.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:31 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.483 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:43:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:31 smithi179 ceph-mon[33633]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 499 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:31 smithi179 ceph-mon[33633]: osdmap e32: 5 total, 5 up, 5 in 2024-02-14T17:43:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:31 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:31 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:31 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:31.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:31 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:31.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:31 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:31.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:31 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:31.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:31 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:32.435 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:32 smithi179 ceph-mon[33633]: Detected new or changed devices on smithi179 2024-02-14T17:43:32.435 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:32 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi179 to 17088M 2024-02-14T17:43:32.436 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:32 smithi179 ceph-mon[33633]: osdmap e33: 5 total, 5 up, 5 in 2024-02-14T17:43:32.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:32 smithi100 ceph-mon[39557]: Detected new or changed devices on smithi179 2024-02-14T17:43:32.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:32 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi179 to 17088M 2024-02-14T17:43:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:32 smithi100 ceph-mon[39557]: osdmap e33: 5 total, 5 up, 5 in 2024-02-14T17:43:32.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:32 smithi100 ceph-mon[30138]: Detected new or changed devices on smithi179 2024-02-14T17:43:32.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:32 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi179 to 17088M 2024-02-14T17:43:32.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:32 smithi100 ceph-mon[30138]: osdmap e33: 5 total, 5 up, 5 in 2024-02-14T17:43:32.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.926+0000 7fcd6c54d700 1 -- 172.21.15.179:0/3632461647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd64103860 msgr2=0x7fcd6410a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:32.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.926+0000 7fcd6c54d700 1 --2- 172.21.15.179:0/3632461647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd64103860 0x7fcd6410a110 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd540099c0 tx=0x7fcd54009cd0 comp rx=0 tx=0).stop 2024-02-14T17:43:32.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.926+0000 7fcd6c54d700 1 -- 172.21.15.179:0/3632461647 shutdown_connections 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.926+0000 7fcd6c54d700 1 --2- 172.21.15.179:0/3632461647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd64103860 0x7fcd6410a110 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.926+0000 7fcd6c54d700 1 --2- 172.21.15.179:0/3632461647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd64101cc0 0x7fcd641020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.926+0000 7fcd6c54d700 1 --2- 172.21.15.179:0/3632461647 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd64102ec0 0x7fcd64103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.926+0000 7fcd6c54d700 1 -- 172.21.15.179:0/3632461647 >> 172.21.15.179:0/3632461647 conn(0x7fcd640fd280 msgr2=0x7fcd640ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.926+0000 7fcd6c54d700 1 -- 172.21.15.179:0/3632461647 shutdown_connections 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.926+0000 7fcd6c54d700 1 -- 172.21.15.179:0/3632461647 wait complete. 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6c54d700 1 Processor -- start 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6c54d700 1 -- start start 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6c54d700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd64101cc0 0x7fcd640759c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6c54d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd64102ec0 0x7fcd64074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6c54d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd64103860 0x7fcd640745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6c54d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd64074d40 con 0x7fcd64103860 2024-02-14T17:43:32.931 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6c54d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd64074eb0 con 0x7fcd64102ec0 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6c54d700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd641a11a0 con 0x7fcd64101cc0 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd69ae8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd64102ec0 0x7fcd64074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd69ae8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd64102ec0 0x7fcd64074060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:48520/0 (socket says 172.21.15.179:48520) 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd69ae8700 1 -- 172.21.15.179:0/1880594577 learned_addr learned my addr 172.21.15.179:0/1880594577 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6aaea700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd64103860 0x7fcd640745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6a2e9700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd64101cc0 0x7fcd640759c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd69ae8700 1 -- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd64101cc0 msgr2=0x7fcd640759c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd69ae8700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd64101cc0 0x7fcd640759c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd69ae8700 1 -- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd64103860 msgr2=0x7fcd640745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd69ae8700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd64103860 0x7fcd640745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd69ae8700 1 -- 172.21.15.179:0/1880594577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540095f0 con 0x7fcd64102ec0 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6aaea700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd64103860 0x7fcd640745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T17:43:32.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd6a2e9700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd64101cc0 0x7fcd640759c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T17:43:32.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.927+0000 7fcd69ae8700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd64102ec0 0x7fcd64074060 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800ef10 tx=0x7fcd5800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:43:32.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.928+0000 7fcd637fe700 1 -- 172.21.15.179:0/1880594577 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5800cad0 con 0x7fcd64102ec0 2024-02-14T17:43:32.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.928+0000 7fcd637fe700 1 -- 172.21.15.179:0/1880594577 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcd5800f040 con 0x7fcd64102ec0 2024-02-14T17:43:32.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.928+0000 7fcd6c54d700 1 -- 172.21.15.179:0/1880594577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd641a13d0 con 0x7fcd64102ec0 2024-02-14T17:43:32.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.928+0000 7fcd637fe700 1 -- 172.21.15.179:0/1880594577 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58014480 con 0x7fcd64102ec0 2024-02-14T17:43:32.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.928+0000 7fcd6c54d700 1 -- 172.21.15.179:0/1880594577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd641a18d0 con 0x7fcd64102ec0 2024-02-14T17:43:32.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.929+0000 7fcd6c54d700 1 -- 172.21.15.179:0/1880594577 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd641088a0 con 0x7fcd64102ec0 2024-02-14T17:43:32.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.929+0000 7fcd637fe700 1 -- 172.21.15.179:0/1880594577 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7fcd58007500 con 0x7fcd64102ec0 2024-02-14T17:43:32.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.929+0000 7fcd637fe700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcd4c06d9d0 0x7fcd4c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:32.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.930+0000 7fcd637fe700 1 -- 172.21.15.179:0/1880594577 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 3991+0+0 (secure 0 0 0) 0x7fcd5808cdf0 con 0x7fcd64102ec0 2024-02-14T17:43:32.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.930+0000 7fcd6a2e9700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcd4c06d9d0 0x7fcd4c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:32.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.930+0000 7fcd6a2e9700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcd4c06d9d0 0x7fcd4c06fe90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd64102d20 tx=0x7fcd5c009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:43:32.936 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:32.934+0000 7fcd637fe700 1 -- 172.21.15.179:0/1880594577 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd58056fc0 con 0x7fcd64102ec0 2024-02-14T17:43:33.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:33.134+0000 7fcd6c54d700 1 -- 172.21.15.179:0/1880594577 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd640620b0 con 0x7fcd4c06d9d0 2024-02-14T17:43:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:33 smithi179 ceph-mon[33633]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 499 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:33 smithi179 ceph-mon[33633]: osdmap e34: 5 total, 5 up, 5 in 2024-02-14T17:43:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:33 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:43:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:33 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:43:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:33 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:33.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:33 smithi100 ceph-mon[39557]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 499 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:33.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:33 smithi100 ceph-mon[39557]: osdmap e34: 5 total, 5 up, 5 in 2024-02-14T17:43:33.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:33 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:43:33.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:33 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:43:33.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:33 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:33 smithi100 ceph-mon[30138]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 499 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:33 smithi100 ceph-mon[30138]: osdmap e34: 5 total, 5 up, 5 in 2024-02-14T17:43:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:33 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:43:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:33 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:43:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:33 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:34.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:34 smithi179 ceph-mon[33633]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:43:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:34 smithi100 ceph-mon[39557]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:43:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:34 smithi100 ceph-mon[30138]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:43:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:35 smithi179 ceph-mon[33633]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-14T17:43:35.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:35 smithi100 ceph-mon[30138]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-14T17:43:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:35 smithi100 ceph-mon[39557]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-02-14T17:43:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/1467787636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60fca708-b509-4b49-823b-e9c81ff48e6d"}]: dispatch 2024-02-14T17:43:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60fca708-b509-4b49-823b-e9c81ff48e6d"}]: dispatch 2024-02-14T17:43:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60fca708-b509-4b49-823b-e9c81ff48e6d"}]': finished 2024-02-14T17:43:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:36 smithi179 ceph-mon[33633]: osdmap e35: 6 total, 5 up, 6 in 2024-02-14T17:43:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:36 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/1467787636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60fca708-b509-4b49-823b-e9c81ff48e6d"}]: dispatch 2024-02-14T17:43:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60fca708-b509-4b49-823b-e9c81ff48e6d"}]: dispatch 2024-02-14T17:43:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60fca708-b509-4b49-823b-e9c81ff48e6d"}]': finished 2024-02-14T17:43:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:36 smithi100 ceph-mon[30138]: osdmap e35: 6 total, 5 up, 6 in 2024-02-14T17:43:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:36 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/1467787636' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60fca708-b509-4b49-823b-e9c81ff48e6d"}]: dispatch 2024-02-14T17:43:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60fca708-b509-4b49-823b-e9c81ff48e6d"}]: dispatch 2024-02-14T17:43:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60fca708-b509-4b49-823b-e9c81ff48e6d"}]': finished 2024-02-14T17:43:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:36 smithi100 ceph-mon[39557]: osdmap e35: 6 total, 5 up, 6 in 2024-02-14T17:43:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:36 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:37 smithi179 ceph-mon[33633]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-02-14T17:43:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/211174593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:43:37.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:37 smithi100 ceph-mon[39557]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-02-14T17:43:37.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/211174593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:43:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:37 smithi100 ceph-mon[30138]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-02-14T17:43:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/211174593' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:43:39.299 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:39 smithi179 ceph-mon[33633]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-14T17:43:39.453 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:39 smithi100 ceph-mon[39557]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-14T17:43:39.453 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:39 smithi100 ceph-mon[30138]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-02-14T17:43:41.418 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:41 smithi179 ceph-mon[33633]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:43:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:41 smithi100 ceph-mon[39557]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:43:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:41 smithi100 ceph-mon[30138]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:43:43.470 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:43 smithi179 ceph-mon[33633]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T17:43:43.470 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-14T17:43:43.470 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:43 smithi100 ceph-mon[39557]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T17:43:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-14T17:43:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:43 smithi100 ceph-mon[30138]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T17:43:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-14T17:43:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:44.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:44 smithi179 ceph-mon[33633]: Deploying daemon osd.5 on smithi179 2024-02-14T17:43:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:44 smithi100 ceph-mon[39557]: Deploying daemon osd.5 on smithi179 2024-02-14T17:43:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:44 smithi100 ceph-mon[30138]: Deploying daemon osd.5 on smithi179 2024-02-14T17:43:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:45 smithi179 ceph-mon[33633]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:45.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:45 smithi100 ceph-mon[39557]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:45 smithi100 ceph-mon[30138]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:47.311 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:47 smithi179 ceph-mon[33633]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:47 smithi100 ceph-mon[30138]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:47.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:47 smithi100 ceph-mon[39557]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:48.714 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:48 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:48.714 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:48 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:48.714 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:48 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:48.715 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:48 smithi179 ceph-mon[33633]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:48 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:48 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:48 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:48 smithi100 ceph-mon[39557]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:48.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:48 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:48.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:48 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:48.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:48 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:48.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:48 smithi100 ceph-mon[30138]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:51.270 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:51 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:51.270 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:51 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:51.270 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:51 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:51.270 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:51 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:51.270 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:51 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:51.270 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:51 smithi179 ceph-mon[33633]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:51.270 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:51 smithi179 ceph-mon[33633]: from='osd.5 [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:43:51.270 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:51 smithi179 ceph-mon[33633]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:43:51.349 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.347+0000 7fcd637fe700 1 -- 172.21.15.179:0/1880594577 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcd640620b0 con 0x7fcd4c06d9d0 2024-02-14T17:43:51.350 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 5 on host 'smithi179' 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 -- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcd4c06d9d0 msgr2=0x7fcd4c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcd4c06d9d0 0x7fcd4c06fe90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd64102d20 tx=0x7fcd5c009670 comp rx=0 tx=0).stop 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 -- 172.21.15.179:0/1880594577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd64102ec0 msgr2=0x7fcd64074060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd64102ec0 0x7fcd64074060 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800ef10 tx=0x7fcd5800c3a0 comp rx=0 tx=0).stop 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 -- 172.21.15.179:0/1880594577 shutdown_connections 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcd64103860 0x7fcd640745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcd64101cc0 0x7fcd640759c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcd4c06d9d0 0x7fcd4c06fe90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 --2- 172.21.15.179:0/1880594577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd64102ec0 0x7fcd64074060 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 -- 172.21.15.179:0/1880594577 >> 172.21.15.179:0/1880594577 conn(0x7fcd640fd280 msgr2=0x7fcd640fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:51.352 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 -- 172.21.15.179:0/1880594577 shutdown_connections 2024-02-14T17:43:51.353 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:51.350+0000 7fcd6c54d700 1 -- 172.21.15.179:0/1880594577 wait complete. 2024-02-14T17:43:51.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[39557]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[39557]: from='osd.5 [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[39557]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:51.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:51.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:51.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[30138]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:51.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[30138]: from='osd.5 [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:43:51.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:51 smithi100 ceph-mon[30138]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-14T17:43:52.017 DEBUG:teuthology.orchestra.run.smithi179:osd.5> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.5.service 2024-02-14T17:43:52.020 INFO:tasks.cephadm:Deploying osd.6 on smithi179 with /dev/vg_nvme/lv_2... 2024-02-14T17:43:52.020 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-14T17:43:52.042 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[39557]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-14T17:43:52.042 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[39557]: osdmap e36: 6 total, 5 up, 6 in 2024-02-14T17:43:52.042 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:52.042 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[39557]: from='osd.5 [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:52.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[30138]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-14T17:43:52.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[30138]: osdmap e36: 6 total, 5 up, 6 in 2024-02-14T17:43:52.042 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:52.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[30138]: from='osd.5 [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:52.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[30138]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:52.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:52.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:52.043 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:52.267 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 17:43:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T17:43:52.053+0000 7f59f29f6700 -1 osd.5 0 waiting for initial osdmap 2024-02-14T17:43:52.267 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 17:43:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T17:43:52.056+0000 7f59ee7e9700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:43:52.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:52 smithi179 ceph-mon[33633]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-14T17:43:52.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:52 smithi179 ceph-mon[33633]: osdmap e36: 6 total, 5 up, 6 in 2024-02-14T17:43:52.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:52 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:52.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:52 smithi179 ceph-mon[33633]: from='osd.5 [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:52.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:52 smithi179 ceph-mon[33633]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:52.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:52 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:52.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:52 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:52.268 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:52 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:52.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[39557]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:43:52.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:52.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:52.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:52 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:43:53.107 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:53 smithi179 ceph-mon[33633]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:43:53.107 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:53 smithi179 ceph-mon[33633]: osdmap e37: 6 total, 5 up, 6 in 2024-02-14T17:43:53.107 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:53 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:53.107 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:53 smithi179 ceph-mon[33633]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:53.107 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:53 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:53.359 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:43:53.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:53 smithi100 ceph-mon[30138]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:43:53.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:53 smithi100 ceph-mon[30138]: osdmap e37: 6 total, 5 up, 6 in 2024-02-14T17:43:53.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:53 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:53.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:53 smithi100 ceph-mon[30138]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:53.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:53 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:53.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:53 smithi100 ceph-mon[39557]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:43:53.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:53 smithi100 ceph-mon[39557]: osdmap e37: 6 total, 5 up, 6 in 2024-02-14T17:43:53.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:53 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:53.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:53 smithi100 ceph-mon[39557]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:53.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:53 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:54.281 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:54 smithi179 ceph-mon[33633]: purged_snaps scrub starts 2024-02-14T17:43:54.281 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:54 smithi179 ceph-mon[33633]: purged_snaps scrub ok 2024-02-14T17:43:54.282 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:54.282 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:54 smithi179 ceph-mon[33633]: osd.5 [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186] boot 2024-02-14T17:43:54.282 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:54 smithi179 ceph-mon[33633]: osdmap e38: 6 total, 6 up, 6 in 2024-02-14T17:43:54.282 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[30138]: purged_snaps scrub starts 2024-02-14T17:43:54.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[30138]: purged_snaps scrub ok 2024-02-14T17:43:54.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:54.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[30138]: osd.5 [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186] boot 2024-02-14T17:43:54.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[30138]: osdmap e38: 6 total, 6 up, 6 in 2024-02-14T17:43:54.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:54.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[39557]: purged_snaps scrub starts 2024-02-14T17:43:54.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[39557]: purged_snaps scrub ok 2024-02-14T17:43:54.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:54.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[39557]: osd.5 [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186] boot 2024-02-14T17:43:54.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[39557]: osdmap e38: 6 total, 6 up, 6 in 2024-02-14T17:43:54.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:43:55.273 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:55 smithi179 ceph-mon[33633]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:55.274 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:55 smithi179 ceph-mon[33633]: osdmap e39: 6 total, 6 up, 6 in 2024-02-14T17:43:55.274 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:55 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.274 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:55 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.274 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:55 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:55.274 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:55 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:55.274 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:55 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.274 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:55 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:55.274 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:55 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:55.274 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:55 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[30138]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[30138]: osdmap e39: 6 total, 6 up, 6 in 2024-02-14T17:43:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:55.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:55.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:55.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:55.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[39557]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-14T17:43:55.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[39557]: osdmap e39: 6 total, 6 up, 6 in 2024-02-14T17:43:55.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:55.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:43:55.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:55.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:55.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:43:55.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:55 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:43:56.125 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-14T17:43:56.125 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-14T17:43:56.125 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-02-14T17:43:56.125 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-02-14T17:43:56.125 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110783 s, 947 MB/s 2024-02-14T17:43:56.125 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-02-14T17:43:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:56 smithi100 ceph-mon[30138]: Detected new or changed devices on smithi179 2024-02-14T17:43:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:56 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi179 to 8544M 2024-02-14T17:43:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:56 smithi100 ceph-mon[30138]: osdmap e40: 6 total, 6 up, 6 in 2024-02-14T17:43:56.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:56 smithi100 ceph-mon[39557]: Detected new or changed devices on smithi179 2024-02-14T17:43:56.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:56 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi179 to 8544M 2024-02-14T17:43:56.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:56 smithi100 ceph-mon[39557]: osdmap e40: 6 total, 6 up, 6 in 2024-02-14T17:43:56.464 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:56 smithi179 ceph-mon[33633]: Detected new or changed devices on smithi179 2024-02-14T17:43:56.464 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:56 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi179 to 8544M 2024-02-14T17:43:56.464 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:56 smithi179 ceph-mon[33633]: osdmap e40: 6 total, 6 up, 6 in 2024-02-14T17:43:56.787 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi179:vg_nvme/lv_2 2024-02-14T17:43:57.025 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:43:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:57 smithi100 ceph-mon[30138]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:43:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:57 smithi100 ceph-mon[30138]: osdmap e41: 6 total, 6 up, 6 in 2024-02-14T17:43:57.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:57 smithi100 ceph-mon[39557]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:43:57.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:57 smithi100 ceph-mon[39557]: osdmap e41: 6 total, 6 up, 6 in 2024-02-14T17:43:57.418 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:57 smithi179 ceph-mon[33633]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:43:57.418 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:57 smithi179 ceph-mon[33633]: osdmap e41: 6 total, 6 up, 6 in 2024-02-14T17:43:58.317 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:58 smithi179 ceph-mon[33633]: osdmap e42: 6 total, 6 up, 6 in 2024-02-14T17:43:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:58 smithi100 ceph-mon[30138]: osdmap e42: 6 total, 6 up, 6 in 2024-02-14T17:43:58.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:58 smithi100 ceph-mon[39557]: osdmap e42: 6 total, 6 up, 6 in 2024-02-14T17:43:58.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.530+0000 7fefa1457700 1 -- 172.21.15.179:0/621445841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef9c105050 msgr2=0x7fef9c107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:58.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.530+0000 7fefa1457700 1 --2- 172.21.15.179:0/621445841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef9c105050 0x7fef9c107440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef84009990 tx=0x7fef84009ca0 comp rx=0 tx=0).stop 2024-02-14T17:43:58.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.530+0000 7fefa1457700 1 -- 172.21.15.179:0/621445841 shutdown_connections 2024-02-14T17:43:58.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.530+0000 7fefa1457700 1 --2- 172.21.15.179:0/621445841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef9c107980 0x7fef9c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.530+0000 7fefa1457700 1 --2- 172.21.15.179:0/621445841 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef9c102720 0x7fef9c104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.530+0000 7fefa1457700 1 --2- 172.21.15.179:0/621445841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef9c105050 0x7fef9c107440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.530+0000 7fefa1457700 1 -- 172.21.15.179:0/621445841 >> 172.21.15.179:0/621445841 conn(0x7fef9c0fc150 msgr2=0x7fef9c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.530+0000 7fefa1457700 1 -- 172.21.15.179:0/621445841 shutdown_connections 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.530+0000 7fefa1457700 1 -- 172.21.15.179:0/621445841 wait complete. 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.530+0000 7fefa1457700 1 Processor -- start 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fefa1457700 1 -- start start 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fefa1457700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef9c102720 0x7fef9c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fefa1457700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef9c105050 0x7fef9c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fefa1457700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef9c107980 0x7fef9c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fefa1457700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c19ed40 con 0x7fef9c102720 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fefa1457700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c19ee80 con 0x7fef9c107980 2024-02-14T17:43:58.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fefa1457700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c19f010 con 0x7fef9c105050 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fef9b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef9c107980 0x7fef9c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fef9b7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef9c107980 0x7fef9c19e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:53266/0 (socket says 172.21.15.179:53266) 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fef9b7fe700 1 -- 172.21.15.179:0/295368723 learned_addr learned my addr 172.21.15.179:0/295368723 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fef9affd700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef9c102720 0x7fef9c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fef927fc700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef9c105050 0x7fef9c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fef9b7fe700 1 -- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef9c105050 msgr2=0x7fef9c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fef9b7fe700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef9c105050 0x7fef9c1976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fef9b7fe700 1 -- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef9c102720 msgr2=0x7fef9c1971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fef9b7fe700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef9c102720 0x7fef9c1971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.531+0000 7fef9b7fe700 1 -- 172.21.15.179:0/295368723 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840095f0 con 0x7fef9c107980 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.532+0000 7fef9b7fe700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef9c107980 0x7fef9c19e770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef940094f0 tx=0x7fef9400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.532+0000 7fef98ff9700 1 -- 172.21.15.179:0/295368723 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94004db0 con 0x7fef9c107980 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.532+0000 7fef98ff9700 1 -- 172.21.15.179:0/295368723 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fef9400f040 con 0x7fef9c107980 2024-02-14T17:43:58.702 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.532+0000 7fefa1457700 1 -- 172.21.15.179:0/295368723 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c19f2f0 con 0x7fef9c107980 2024-02-14T17:43:58.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.532+0000 7fef98ff9700 1 -- 172.21.15.179:0/295368723 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef94005870 con 0x7fef9c107980 2024-02-14T17:43:58.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.532+0000 7fefa1457700 1 -- 172.21.15.179:0/295368723 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c19f7a0 con 0x7fef9c107980 2024-02-14T17:43:58.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.533+0000 7fefa1457700 1 -- 172.21.15.179:0/295368723 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9c0fdcd0 con 0x7fef9c107980 2024-02-14T17:43:58.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.533+0000 7fef98ff9700 1 -- 172.21.15.179:0/295368723 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7fef9400db20 con 0x7fef9c107980 2024-02-14T17:43:58.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.534+0000 7fef98ff9700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fef8806d9d0 0x7fef8806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:43:58.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.534+0000 7fef98ff9700 1 -- 172.21.15.179:0/295368723 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4423+0+0 (secure 0 0 0) 0x7fef9408c400 con 0x7fef9c107980 2024-02-14T17:43:58.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.534+0000 7fef9affd700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fef8806d9d0 0x7fef8806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:43:58.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.534+0000 7fef9affd700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fef8806d9d0 0x7fef8806fe90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fef8c004630 tx=0x7fef8c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:43:58.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.538+0000 7fef98ff9700 1 -- 172.21.15.179:0/295368723 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef94056420 con 0x7fef9c107980 2024-02-14T17:43:58.741 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:43:58.740+0000 7fefa1457700 1 -- 172.21.15.179:0/295368723 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fef9c0620b0 con 0x7fef8806d9d0 2024-02-14T17:43:59.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:59 smithi100 ceph-mon[30138]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:43:59.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:43:59.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:43:59.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:43:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:59.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:59 smithi100 ceph-mon[39557]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:43:59.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:59 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:43:59.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:59 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:43:59.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:43:59 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:43:59.416 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:59 smithi179 ceph-mon[33633]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:43:59.416 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:43:59.416 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:43:59.416 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:43:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:00.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:00 smithi179 ceph-mon[33633]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:44:00.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:00 smithi100 ceph-mon[30138]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:44:00.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:00 smithi100 ceph-mon[39557]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:44:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:01 smithi100 ceph-mon[30138]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:01.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:01 smithi100 ceph-mon[39557]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:01 smithi179 ceph-mon[33633]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:02.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/2334062284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d797fe1-a6df-4e3d-a923-8ebbafa7a49b"}]: dispatch 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d797fe1-a6df-4e3d-a923-8ebbafa7a49b"}]: dispatch 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d797fe1-a6df-4e3d-a923-8ebbafa7a49b"}]': finished 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[30138]: osdmap e43: 7 total, 6 up, 7 in 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/4133406184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/2334062284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d797fe1-a6df-4e3d-a923-8ebbafa7a49b"}]: dispatch 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d797fe1-a6df-4e3d-a923-8ebbafa7a49b"}]: dispatch 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d797fe1-a6df-4e3d-a923-8ebbafa7a49b"}]': finished 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[39557]: osdmap e43: 7 total, 6 up, 7 in 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:02.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/4133406184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:44:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/2334062284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d797fe1-a6df-4e3d-a923-8ebbafa7a49b"}]: dispatch 2024-02-14T17:44:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d797fe1-a6df-4e3d-a923-8ebbafa7a49b"}]: dispatch 2024-02-14T17:44:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d797fe1-a6df-4e3d-a923-8ebbafa7a49b"}]': finished 2024-02-14T17:44:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:02 smithi179 ceph-mon[33633]: osdmap e43: 7 total, 6 up, 7 in 2024-02-14T17:44:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:02 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/4133406184' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:44:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:03 smithi100 ceph-mon[30138]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:03 smithi100 ceph-mon[39557]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:03 smithi179 ceph-mon[33633]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:05 smithi179 ceph-mon[33633]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:05 smithi100 ceph-mon[30138]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:05 smithi100 ceph-mon[39557]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:07 smithi179 ceph-mon[33633]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:07 smithi100 ceph-mon[30138]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:07 smithi100 ceph-mon[39557]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-14T17:44:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:08 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:08.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-14T17:44:08.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:08 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:08.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-14T17:44:08.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:08 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:09 smithi100 ceph-mon[30138]: Deploying daemon osd.6 on smithi179 2024-02-14T17:44:09.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:09 smithi100 ceph-mon[30138]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:09.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:09 smithi100 ceph-mon[39557]: Deploying daemon osd.6 on smithi179 2024-02-14T17:44:09.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:09 smithi100 ceph-mon[39557]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:09.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:09 smithi179 ceph-mon[33633]: Deploying daemon osd.6 on smithi179 2024-02-14T17:44:09.517 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:09 smithi179 ceph-mon[33633]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:11.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:11 smithi100 ceph-mon[39557]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:11.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:11 smithi100 ceph-mon[30138]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:11.402 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:11 smithi179 ceph-mon[33633]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:13 smithi100 ceph-mon[39557]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:13 smithi100 ceph-mon[30138]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:13 smithi179 ceph-mon[33633]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:14 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:14 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:14 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:14.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:14 smithi179 ceph-mon[33633]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:14 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:14 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:14 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:14.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:14 smithi100 ceph-mon[39557]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:14.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:14 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:14.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:14 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:14.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:14 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:14.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:14 smithi100 ceph-mon[30138]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:17.051 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.049+0000 7fef98ff9700 1 -- 172.21.15.179:0/295368723 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fef9c0620b0 con 0x7fef8806d9d0 2024-02-14T17:44:17.051 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 6 on host 'smithi179' 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 -- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fef8806d9d0 msgr2=0x7fef8806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fef8806d9d0 0x7fef8806fe90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fef8c004630 tx=0x7fef8c00b3c0 comp rx=0 tx=0).stop 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 -- 172.21.15.179:0/295368723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef9c107980 msgr2=0x7fef9c19e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef9c107980 0x7fef9c19e770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fef940094f0 tx=0x7fef9400d3b0 comp rx=0 tx=0).stop 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 -- 172.21.15.179:0/295368723 shutdown_connections 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fef8806d9d0 0x7fef8806fe90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef9c107980 0x7fef9c19e770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fef9c102720 0x7fef9c1971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 --2- 172.21.15.179:0/295368723 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fef9c105050 0x7fef9c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 -- 172.21.15.179:0/295368723 >> 172.21.15.179:0/295368723 conn(0x7fef9c0fc150 msgr2=0x7fef9c103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 -- 172.21.15.179:0/295368723 shutdown_connections 2024-02-14T17:44:17.054 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:17.052+0000 7fefa1457700 1 -- 172.21.15.179:0/295368723 wait complete. 2024-02-14T17:44:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: from='osd.6 [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T17:44:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T17:44:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.636 DEBUG:teuthology.orchestra.run.smithi179:osd.6> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.6.service 2024-02-14T17:44:17.638 INFO:tasks.cephadm:Deploying osd.7 on smithi179 with /dev/vg_nvme/lv_1... 2024-02-14T17:44:17.639 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-14T17:44:17.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:17.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: from='osd.6 [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T17:44:17.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T17:44:17.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:17.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: from='osd.6 [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:17.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:17.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:18.391 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:18 smithi179 ceph-mon[33633]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-14T17:44:18.391 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:18 smithi179 ceph-mon[33633]: osdmap e44: 7 total, 6 up, 7 in 2024-02-14T17:44:18.391 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:18 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:18.391 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:18 smithi179 ceph-mon[33633]: from='osd.6 [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:18.391 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:18 smithi179 ceph-mon[33633]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:18.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:18 smithi100 ceph-mon[30138]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-14T17:44:18.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:18 smithi100 ceph-mon[30138]: osdmap e44: 7 total, 6 up, 7 in 2024-02-14T17:44:18.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:18.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:18 smithi100 ceph-mon[30138]: from='osd.6 [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:18.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:18 smithi100 ceph-mon[30138]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:18.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:18 smithi100 ceph-mon[39557]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-14T17:44:18.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:18 smithi100 ceph-mon[39557]: osdmap e44: 7 total, 6 up, 7 in 2024-02-14T17:44:18.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:18 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:18.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:18 smithi100 ceph-mon[39557]: from='osd.6 [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:18.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:18 smithi100 ceph-mon[39557]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:18.803 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 17:44:18 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T17:44:18.599+0000 7f51a0310700 -1 osd.6 0 waiting for initial osdmap 2024-02-14T17:44:18.804 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 17:44:18 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T17:44:18.606+0000 7f51980fb700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:44:18.821 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:44:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:19 smithi179 ceph-mon[33633]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:19 smithi179 ceph-mon[33633]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:44:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:19 smithi179 ceph-mon[33633]: osdmap e45: 7 total, 6 up, 7 in 2024-02-14T17:44:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:19 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:19 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:19 smithi179 ceph-mon[33633]: from='osd.6 ' entity='osd.6' 2024-02-14T17:44:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:19 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:19.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[30138]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:19.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[30138]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:44:19.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[30138]: osdmap e45: 7 total, 6 up, 7 in 2024-02-14T17:44:19.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:19.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:19.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[30138]: from='osd.6 ' entity='osd.6' 2024-02-14T17:44:19.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:19.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[39557]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:19.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[39557]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:44:19.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[39557]: osdmap e45: 7 total, 6 up, 7 in 2024-02-14T17:44:19.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:19.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:19.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[39557]: from='osd.6 ' entity='osd.6' 2024-02-14T17:44:19.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:19 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:20 smithi179 ceph-mon[33633]: purged_snaps scrub starts 2024-02-14T17:44:20.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:20 smithi179 ceph-mon[33633]: purged_snaps scrub ok 2024-02-14T17:44:20.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:20 smithi179 ceph-mon[33633]: osd.6 [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257] boot 2024-02-14T17:44:20.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:20 smithi179 ceph-mon[33633]: osdmap e46: 7 total, 7 up, 7 in 2024-02-14T17:44:20.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:20 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:20.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:20 smithi179 ceph-mon[33633]: osdmap e47: 7 total, 7 up, 7 in 2024-02-14T17:44:20.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[30138]: purged_snaps scrub starts 2024-02-14T17:44:20.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[30138]: purged_snaps scrub ok 2024-02-14T17:44:20.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[30138]: osd.6 [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257] boot 2024-02-14T17:44:20.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[30138]: osdmap e46: 7 total, 7 up, 7 in 2024-02-14T17:44:20.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:20.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[30138]: osdmap e47: 7 total, 7 up, 7 in 2024-02-14T17:44:20.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[39557]: purged_snaps scrub starts 2024-02-14T17:44:20.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[39557]: purged_snaps scrub ok 2024-02-14T17:44:20.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[39557]: osd.6 [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257] boot 2024-02-14T17:44:20.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[39557]: osdmap e46: 7 total, 7 up, 7 in 2024-02-14T17:44:20.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:44:20.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:20 smithi100 ceph-mon[39557]: osdmap e47: 7 total, 7 up, 7 in 2024-02-14T17:44:21.223 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:21 smithi179 ceph-mon[33633]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:21.223 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:21 smithi179 ceph-mon[33633]: osdmap e48: 7 total, 7 up, 7 in 2024-02-14T17:44:21.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:21 smithi100 ceph-mon[39557]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:21.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:21 smithi100 ceph-mon[39557]: osdmap e48: 7 total, 7 up, 7 in 2024-02-14T17:44:21.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:21 smithi100 ceph-mon[30138]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-14T17:44:21.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:21 smithi100 ceph-mon[30138]: osdmap e48: 7 total, 7 up, 7 in 2024-02-14T17:44:21.803 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-14T17:44:21.803 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-14T17:44:21.803 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-02-14T17:44:21.803 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-02-14T17:44:21.803 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112322 s, 934 MB/s 2024-02-14T17:44:21.803 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: osdmap e49: 7 total, 7 up, 7 in 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: Detected new or changed devices on smithi179 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi179 to 5696M 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:22.413 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.414 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:22 smithi179 ceph-mon[33633]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:22.440 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi179:vg_nvme/lv_1 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: osdmap e49: 7 total, 7 up, 7 in 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: Detected new or changed devices on smithi179 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi179 to 5696M 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[30138]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: osdmap e49: 7 total, 7 up, 7 in 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: Detected new or changed devices on smithi179 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi179 to 5696M 2024-02-14T17:44:22.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:22.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:22.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:22.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:22 smithi100 ceph-mon[39557]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:22.679 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:44:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:23 smithi179 ceph-mon[33633]: osdmap e50: 7 total, 7 up, 7 in 2024-02-14T17:44:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:23 smithi100 ceph-mon[30138]: osdmap e50: 7 total, 7 up, 7 in 2024-02-14T17:44:23.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:23 smithi100 ceph-mon[39557]: osdmap e50: 7 total, 7 up, 7 in 2024-02-14T17:44:24.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.236+0000 7fae7cf92700 1 -- 172.21.15.179:0/1648849958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae781017b0 msgr2=0x7fae78109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:24.238 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.236+0000 7fae7cf92700 1 --2- 172.21.15.179:0/1648849958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae781017b0 0x7fae78109050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b0a0 tx=0x7fae7000b3b0 comp rx=0 tx=0).stop 2024-02-14T17:44:24.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.236+0000 7fae7cf92700 1 -- 172.21.15.179:0/1648849958 shutdown_connections 2024-02-14T17:44:24.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.236+0000 7fae7cf92700 1 --2- 172.21.15.179:0/1648849958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae781017b0 0x7fae78109050 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:24.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.236+0000 7fae7cf92700 1 --2- 172.21.15.179:0/1648849958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae78102220 0x7fae78102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:24.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.236+0000 7fae7cf92700 1 --2- 172.21.15.179:0/1648849958 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae78100df0 0x7fae78101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:24.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.236+0000 7fae7cf92700 1 -- 172.21.15.179:0/1648849958 >> 172.21.15.179:0/1648849958 conn(0x7fae780fb8e0 msgr2=0x7fae780fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:24.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.236+0000 7fae7cf92700 1 -- 172.21.15.179:0/1648849958 shutdown_connections 2024-02-14T17:44:24.239 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.236+0000 7fae7cf92700 1 -- 172.21.15.179:0/1648849958 wait complete. 2024-02-14T17:44:24.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7cf92700 1 Processor -- start 2024-02-14T17:44:24.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7cf92700 1 -- start start 2024-02-14T17:44:24.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7cf92700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae78100df0 0x7fae781007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:24.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7cf92700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae781017b0 0x7fae7819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:24.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7cf92700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae78102220 0x7fae7819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:24.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7cf92700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7819edc0 con 0x7fae781017b0 2024-02-14T17:44:24.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7cf92700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7819ef30 con 0x7fae78102220 2024-02-14T17:44:24.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7cf92700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7819f0f0 con 0x7fae78100df0 2024-02-14T17:44:24.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae76d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae78102220 0x7fae7819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7659c700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae78100df0 0x7fae781007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7659c700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae78100df0 0x7fae781007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.179:57232/0 (socket says 172.21.15.179:57232) 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae76d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae78102220 0x7fae7819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51830/0 (socket says 172.21.15.179:51830) 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae7659c700 1 -- 172.21.15.179:0/2646183193 learned_addr learned my addr 172.21.15.179:0/2646183193 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.237+0000 7fae75d9b700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae781017b0 0x7fae7819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae76d9d700 1 -- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae78100df0 msgr2=0x7fae781007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae76d9d700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae78100df0 0x7fae781007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae76d9d700 1 -- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae781017b0 msgr2=0x7fae7819c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae76d9d700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae781017b0 0x7fae7819c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae76d9d700 1 -- 172.21.15.179:0/2646183193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae70009d00 con 0x7fae78102220 2024-02-14T17:44:24.243 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae76d9d700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae78102220 0x7fae7819e620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b070 tx=0x7fae70009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:24.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae677fe700 1 -- 172.21.15.179:0/2646183193 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7000e040 con 0x7fae78102220 2024-02-14T17:44:24.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae7cf92700 1 -- 172.21.15.179:0/2646183193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7819f370 con 0x7fae78102220 2024-02-14T17:44:24.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae677fe700 1 -- 172.21.15.179:0/2646183193 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fae7000be40 con 0x7fae78102220 2024-02-14T17:44:24.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae677fe700 1 -- 172.21.15.179:0/2646183193 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70005040 con 0x7fae78102220 2024-02-14T17:44:24.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.238+0000 7fae7cf92700 1 -- 172.21.15.179:0/2646183193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7819f7e0 con 0x7fae78102220 2024-02-14T17:44:24.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.239+0000 7fae7cf92700 1 -- 172.21.15.179:0/2646183193 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae781077e0 con 0x7fae78102220 2024-02-14T17:44:24.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.240+0000 7fae677fe700 1 -- 172.21.15.179:0/2646183193 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7fae7001d470 con 0x7fae78102220 2024-02-14T17:44:24.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.240+0000 7fae677fe700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fae6006d9d0 0x7fae6006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:24.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.240+0000 7fae7659c700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fae6006d9d0 0x7fae6006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:24.244 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.241+0000 7fae7659c700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fae6006d9d0 0x7fae6006fe90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae68009ca0 tx=0x7fae68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:24.245 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.241+0000 7fae677fe700 1 -- 172.21.15.179:0/2646183193 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 4855+0+0 (secure 0 0 0) 0x7fae7005c450 con 0x7fae78102220 2024-02-14T17:44:24.248 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.246+0000 7fae677fe700 1 -- 172.21.15.179:0/2646183193 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae70057ff0 con 0x7fae78102220 2024-02-14T17:44:24.450 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:24.449+0000 7fae7cf92700 1 -- 172.21.15.179:0/2646183193 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fae7810bbf0 con 0x7fae6006d9d0 2024-02-14T17:44:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:24 smithi179 ceph-mon[33633]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:24 smithi100 ceph-mon[30138]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:24 smithi100 ceph-mon[39557]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:25 smithi179 ceph-mon[33633]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:44:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:25 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:44:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:25 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:44:25.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:25 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:25 smithi100 ceph-mon[30138]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:44:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:25 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:44:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:25 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:44:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:25 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:25 smithi100 ceph-mon[39557]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:44:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:25 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-14T17:44:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:25 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-14T17:44:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:25 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:26.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:26 smithi179 ceph-mon[33633]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T17:44:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:26 smithi100 ceph-mon[30138]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T17:44:26.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:26 smithi100 ceph-mon[39557]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-14T17:44:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/3639644700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51596ca9-64f6-45fd-87ce-3a83bd7b9c14"}]: dispatch 2024-02-14T17:44:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51596ca9-64f6-45fd-87ce-3a83bd7b9c14"}]: dispatch 2024-02-14T17:44:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51596ca9-64f6-45fd-87ce-3a83bd7b9c14"}]': finished 2024-02-14T17:44:27.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:27 smithi179 ceph-mon[33633]: osdmap e51: 8 total, 7 up, 8 in 2024-02-14T17:44:27.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:27 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/3639644700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51596ca9-64f6-45fd-87ce-3a83bd7b9c14"}]: dispatch 2024-02-14T17:44:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51596ca9-64f6-45fd-87ce-3a83bd7b9c14"}]: dispatch 2024-02-14T17:44:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51596ca9-64f6-45fd-87ce-3a83bd7b9c14"}]': finished 2024-02-14T17:44:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:27 smithi100 ceph-mon[30138]: osdmap e51: 8 total, 7 up, 8 in 2024-02-14T17:44:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:27 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/3639644700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51596ca9-64f6-45fd-87ce-3a83bd7b9c14"}]: dispatch 2024-02-14T17:44:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51596ca9-64f6-45fd-87ce-3a83bd7b9c14"}]: dispatch 2024-02-14T17:44:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51596ca9-64f6-45fd-87ce-3a83bd7b9c14"}]': finished 2024-02-14T17:44:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:27 smithi100 ceph-mon[39557]: osdmap e51: 8 total, 7 up, 8 in 2024-02-14T17:44:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:27 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/120707508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:44:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:28 smithi179 ceph-mon[33633]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-14T17:44:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/120707508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:44:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:28 smithi100 ceph-mon[30138]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-14T17:44:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/120707508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-14T17:44:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:28 smithi100 ceph-mon[39557]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-14T17:44:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:31 smithi100 ceph-mon[30138]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:44:31.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:31 smithi100 ceph-mon[39557]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:44:31.454 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:31 smithi179 ceph-mon[33633]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:44:33.199 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:33 smithi179 ceph-mon[33633]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-14T17:44:33.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:33 smithi100 ceph-mon[30138]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-14T17:44:33.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:33 smithi100 ceph-mon[39557]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-14T17:44:34.378 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:34 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-14T17:44:34.378 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:34 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-14T17:44:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:34 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:34.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:34 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-14T17:44:34.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:34 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:35 smithi100 ceph-mon[39557]: Deploying daemon osd.7 on smithi179 2024-02-14T17:44:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:35 smithi100 ceph-mon[39557]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T17:44:35.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:35 smithi100 ceph-mon[30138]: Deploying daemon osd.7 on smithi179 2024-02-14T17:44:35.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:35 smithi100 ceph-mon[30138]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T17:44:35.439 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:35 smithi179 ceph-mon[33633]: Deploying daemon osd.7 on smithi179 2024-02-14T17:44:35.439 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:35 smithi179 ceph-mon[33633]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-14T17:44:37.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:37 smithi100 ceph-mon[30138]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:37 smithi100 ceph-mon[39557]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:37.429 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:37 smithi179 ceph-mon[33633]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:39 smithi100 ceph-mon[30138]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:39 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:39 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:39 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:39.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:39 smithi100 ceph-mon[39557]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:39.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:39 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:39.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:39 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:39.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:39 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:39.411 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:39 smithi179 ceph-mon[33633]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:39.411 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:39 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:39.411 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:39 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:39.411 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:39 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:41.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:41 smithi100 ceph-mon[30138]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:41.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:41 smithi100 ceph-mon[39557]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:41.456 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:41 smithi179 ceph-mon[33633]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:42.317 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:42 smithi179 ceph-mon[33633]: from='osd.7 [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:44:42.317 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:42 smithi179 ceph-mon[33633]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:44:42.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.362+0000 7fae677fe700 1 -- 172.21.15.179:0/2646183193 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fae7810bbf0 con 0x7fae6006d9d0 2024-02-14T17:44:42.365 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 7 on host 'smithi179' 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 -- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fae6006d9d0 msgr2=0x7fae6006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fae6006d9d0 0x7fae6006fe90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae68009ca0 tx=0x7fae68009260 comp rx=0 tx=0).stop 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 -- 172.21.15.179:0/2646183193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae78102220 msgr2=0x7fae7819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae78102220 0x7fae7819e620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b070 tx=0x7fae70009580 comp rx=0 tx=0).stop 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 -- 172.21.15.179:0/2646183193 shutdown_connections 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fae6006d9d0 0x7fae6006fe90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae78102220 0x7fae7819e620 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fae78100df0 0x7fae781007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 --2- 172.21.15.179:0/2646183193 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fae781017b0 0x7fae7819c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 -- 172.21.15.179:0/2646183193 >> 172.21.15.179:0/2646183193 conn(0x7fae780fb8e0 msgr2=0x7fae780fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 -- 172.21.15.179:0/2646183193 shutdown_connections 2024-02-14T17:44:42.367 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:42.365+0000 7fae7cf92700 1 -- 172.21.15.179:0/2646183193 wait complete. 2024-02-14T17:44:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:42 smithi100 ceph-mon[30138]: from='osd.7 [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:44:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:42 smithi100 ceph-mon[30138]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:44:42.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:42 smithi100 ceph-mon[39557]: from='osd.7 [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:44:42.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:42 smithi100 ceph-mon[39557]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-14T17:44:42.981 DEBUG:teuthology.orchestra.run.smithi179:osd.7> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.7.service 2024-02-14T17:44:42.983 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-14T17:44:42.984 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd stat -f json 2024-02-14T17:44:43.239 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:44:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:43 smithi179 ceph-mon[33633]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:43 smithi179 ceph-mon[33633]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-14T17:44:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:43 smithi179 ceph-mon[33633]: from='osd.7 [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:43 smithi179 ceph-mon[33633]: osdmap e52: 8 total, 7 up, 8 in 2024-02-14T17:44:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:43 smithi179 ceph-mon[33633]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:43 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:43.361 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[30138]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:43.361 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[30138]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-14T17:44:43.361 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[30138]: from='osd.7 [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[30138]: osdmap e52: 8 total, 7 up, 8 in 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[30138]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[39557]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[39557]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[39557]: from='osd.7 [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[39557]: osdmap e52: 8 total, 7 up, 8 in 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[39557]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:43.362 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:43 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:43.960 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 17:44:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T17:44:43.596+0000 7fce46f6b700 -1 osd.7 0 waiting for initial osdmap 2024-02-14T17:44:43.960 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 17:44:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T17:44:43.604+0000 7fce3ed56700 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-14T17:44:44.283 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:44 smithi179 ceph-mon[33633]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:44:44.283 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:44 smithi179 ceph-mon[33633]: osdmap e53: 8 total, 7 up, 8 in 2024-02-14T17:44:44.283 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:44.283 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:44.283 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:44.283 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:44.284 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:44 smithi179 ceph-mon[33633]: from='osd.7 ' entity='osd.7' 2024-02-14T17:44:44.284 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:44 smithi179 ceph-mon[33633]: osd.7 [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483] boot 2024-02-14T17:44:44.284 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:44 smithi179 ceph-mon[33633]: osdmap e54: 8 total, 8 up, 8 in 2024-02-14T17:44:44.284 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:44 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:44.574 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[39557]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:44:44.574 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[39557]: osdmap e53: 8 total, 7 up, 8 in 2024-02-14T17:44:44.574 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:44.574 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:44.574 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:44.574 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:44.574 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[39557]: from='osd.7 ' entity='osd.7' 2024-02-14T17:44:44.574 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[39557]: osd.7 [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483] boot 2024-02-14T17:44:44.574 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[39557]: osdmap e54: 8 total, 8 up, 8 in 2024-02-14T17:44:44.574 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:44.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[30138]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-02-14T17:44:44.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[30138]: osdmap e53: 8 total, 7 up, 8 in 2024-02-14T17:44:44.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:44.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:44.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:44.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:44.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[30138]: from='osd.7 ' entity='osd.7' 2024-02-14T17:44:44.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[30138]: osd.7 [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483] boot 2024-02-14T17:44:44.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[30138]: osdmap e54: 8 total, 8 up, 8 in 2024-02-14T17:44:44.575 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:44 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:44:45.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.101+0000 7f72a004e700 1 -- 172.21.15.100:0/3615215896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7298102ea0 msgr2=0x7f7298103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:45.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.101+0000 7f72a004e700 1 --2- 172.21.15.100:0/3615215896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7298102ea0 0x7f7298103300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7294009920 tx=0x7f7294009c30 comp rx=0 tx=0).stop 2024-02-14T17:44:45.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.101+0000 7f72a004e700 1 -- 172.21.15.100:0/3615215896 shutdown_connections 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.101+0000 7f72a004e700 1 --2- 172.21.15.100:0/3615215896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7298103840 0x7f729810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.101+0000 7f72a004e700 1 --2- 172.21.15.100:0/3615215896 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7298101ca0 0x7f72981020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.101+0000 7f72a004e700 1 --2- 172.21.15.100:0/3615215896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7298102ea0 0x7f7298103300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.101+0000 7f72a004e700 1 -- 172.21.15.100:0/3615215896 >> 172.21.15.100:0/3615215896 conn(0x7f72980fd280 msgr2=0x7f72980ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.101+0000 7f72a004e700 1 -- 172.21.15.100:0/3615215896 shutdown_connections 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.101+0000 7f72a004e700 1 -- 172.21.15.100:0/3615215896 wait complete. 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f72a004e700 1 Processor -- start 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f72a004e700 1 -- start start 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f72a004e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7298101ca0 0x7f7298199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f72a004e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7298102ea0 0x7f7298199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f72a004e700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7298103840 0x7f72981a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:45.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f72a004e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72981a10c0 con 0x7f7298101ca0 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f72a004e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72981a1200 con 0x7f7298102ea0 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f72a004e700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72981a1390 con 0x7f7298103840 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f729ddea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7298101ca0 0x7f7298199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f729ddea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7298101ca0 0x7f7298199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54206/0 (socket says 172.21.15.100:54206) 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f729ddea700 1 -- 172.21.15.100:0/1267073746 learned_addr learned my addr 172.21.15.100:0/1267073746 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f729e5eb700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7298103840 0x7f72981a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.102+0000 7f729d5e9700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7298102ea0 0x7f7298199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f729ddea700 1 -- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7298103840 msgr2=0x7f72981a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f729ddea700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7298103840 0x7f72981a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f729ddea700 1 -- 172.21.15.100:0/1267073746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7298102ea0 msgr2=0x7f7298199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f729ddea700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7298102ea0 0x7f7298199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f729ddea700 1 -- 172.21.15.100:0/1267073746 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72880095f0 con 0x7f7298101ca0 2024-02-14T17:44:45.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f729d5e9700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7298102ea0 0x7f7298199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T17:44:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f729e5eb700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7298103840 0x7f72981a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:44:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f729ddea700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7298101ca0 0x7f7298199530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f728800ec20 tx=0x7f728800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f728effd700 1 -- 172.21.15.100:0/1267073746 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288009a10 con 0x7f7298101ca0 2024-02-14T17:44:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f728effd700 1 -- 172.21.15.100:0/1267073746 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f728800cd40 con 0x7f7298101ca0 2024-02-14T17:44:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f728effd700 1 -- 172.21.15.100:0/1267073746 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72880186b0 con 0x7f7298101ca0 2024-02-14T17:44:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f72a004e700 1 -- 172.21.15.100:0/1267073746 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7294009520 con 0x7f7298101ca0 2024-02-14T17:44:45.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.103+0000 7f72a004e700 1 -- 172.21.15.100:0/1267073746 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72981a1ab0 con 0x7f7298101ca0 2024-02-14T17:44:45.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.104+0000 7f72a004e700 1 -- 172.21.15.100:0/1267073746 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7298068930 con 0x7f7298101ca0 2024-02-14T17:44:45.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.104+0000 7f728effd700 1 -- 172.21.15.100:0/1267073746 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7f7288010430 con 0x7f7298101ca0 2024-02-14T17:44:45.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.105+0000 7f728effd700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f728406d9d0 0x7f728406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:45.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.105+0000 7f728effd700 1 -- 172.21.15.100:0/1267073746 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5316+0+0 (secure 0 0 0) 0x7f7288014070 con 0x7f7298101ca0 2024-02-14T17:44:45.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.105+0000 7f729d5e9700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f728406d9d0 0x7f728406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:45.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.105+0000 7f729d5e9700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f728406d9d0 0x7f728406fe90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f729400c010 tx=0x7f7294019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:45.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.109+0000 7f728effd700 1 -- 172.21.15.100:0/1267073746 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f72880571d0 con 0x7f7298101ca0 2024-02-14T17:44:45.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.309+0000 7f72a004e700 1 -- 172.21.15.100:0/1267073746 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f729810cd30 con 0x7f7298101ca0 2024-02-14T17:44:45.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.309+0000 7f728effd700 1 -- 172.21.15.100:0/1267073746 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v56) v1 ==== 74+0+130 (secure 0 0 0) 0x7f728805a890 con 0x7f7298101ca0 2024-02-14T17:44:45.311 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:44:45.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 -- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f728406d9d0 msgr2=0x7f728406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:45.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f728406d9d0 0x7f728406fe90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f729400c010 tx=0x7f7294019040 comp rx=0 tx=0).stop 2024-02-14T17:44:45.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 -- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7298101ca0 msgr2=0x7f7298199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:45.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7298101ca0 0x7f7298199530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f728800ec20 tx=0x7f728800ef30 comp rx=0 tx=0).stop 2024-02-14T17:44:45.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 -- 172.21.15.100:0/1267073746 shutdown_connections 2024-02-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f7298103840 0x7f72981a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7298101ca0 0x7f7298199530 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f728406d9d0 0x7f728406fe90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 --2- 172.21.15.100:0/1267073746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7298102ea0 0x7f7298199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 -- 172.21.15.100:0/1267073746 >> 172.21.15.100:0/1267073746 conn(0x7f72980fd280 msgr2=0x7f72980fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 -- 172.21.15.100:0/1267073746 shutdown_connections 2024-02-14T17:44:45.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:45.311+0000 7f72a004e700 1 -- 172.21.15.100:0/1267073746 wait complete. 2024-02-14T17:44:45.441 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:45 smithi179 ceph-mon[33633]: purged_snaps scrub starts 2024-02-14T17:44:45.442 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:45 smithi179 ceph-mon[33633]: purged_snaps scrub ok 2024-02-14T17:44:45.442 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:45 smithi179 ceph-mon[33633]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:45.442 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:45 smithi179 ceph-mon[33633]: osdmap e55: 8 total, 8 up, 8 in 2024-02-14T17:44:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:45 smithi100 ceph-mon[39557]: purged_snaps scrub starts 2024-02-14T17:44:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:45 smithi100 ceph-mon[39557]: purged_snaps scrub ok 2024-02-14T17:44:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:45 smithi100 ceph-mon[39557]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:45 smithi100 ceph-mon[39557]: osdmap e55: 8 total, 8 up, 8 in 2024-02-14T17:44:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:45 smithi100 ceph-mon[30138]: purged_snaps scrub starts 2024-02-14T17:44:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:45 smithi100 ceph-mon[30138]: purged_snaps scrub ok 2024-02-14T17:44:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:45 smithi100 ceph-mon[30138]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-14T17:44:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:45 smithi100 ceph-mon[30138]: osdmap e55: 8 total, 8 up, 8 in 2024-02-14T17:44:46.115 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1707932684,"num_in_osds":8,"osd_in_since":1707932667,"num_remapped_pgs":1} 2024-02-14T17:44:46.116 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T17:44:46.320 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:46 smithi179 ceph-mon[33633]: osdmap e56: 8 total, 8 up, 8 in 2024-02-14T17:44:46.320 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1267073746' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:44:46.320 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:46 smithi179 ceph-mon[33633]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:46.359 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:44:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:46 smithi100 ceph-mon[30138]: osdmap e56: 8 total, 8 up, 8 in 2024-02-14T17:44:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1267073746' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:44:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:46 smithi100 ceph-mon[30138]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:46.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:46 smithi100 ceph-mon[39557]: osdmap e56: 8 total, 8 up, 8 in 2024-02-14T17:44:46.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1267073746' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-14T17:44:46.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:46 smithi100 ceph-mon[39557]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:47.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: osdmap e57: 8 total, 8 up, 8 in 2024-02-14T17:44:47.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: Detected new or changed devices on smithi179 2024-02-14T17:44:47.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi179 to 4272M 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: osdmap e57: 8 total, 8 up, 8 in 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: Detected new or changed devices on smithi179 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.485 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi179 to 4272M 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:47.486 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:47 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: osdmap e57: 8 total, 8 up, 8 in 2024-02-14T17:44:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: Detected new or changed devices on smithi179 2024-02-14T17:44:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi179 to 4272M 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:47 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:48.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.011+0000 7f46ef1ad700 1 -- 172.21.15.100:0/387131895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e8101520 msgr2=0x7f46e810d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:48.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.011+0000 7f46ef1ad700 1 --2- 172.21.15.100:0/387131895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e8101520 0x7f46e810d9b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f46d80098f0 tx=0x7f46d8009c00 comp rx=0 tx=0).stop 2024-02-14T17:44:48.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.011+0000 7f46ef1ad700 1 -- 172.21.15.100:0/387131895 shutdown_connections 2024-02-14T17:44:48.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.011+0000 7f46ef1ad700 1 --2- 172.21.15.100:0/387131895 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e8101520 0x7f46e810d9b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.011+0000 7f46ef1ad700 1 --2- 172.21.15.100:0/387131895 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e8106be0 0x7f46e8106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.011+0000 7f46ef1ad700 1 --2- 172.21.15.100:0/387131895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e8100b60 0x7f46e8100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.011+0000 7f46ef1ad700 1 -- 172.21.15.100:0/387131895 >> 172.21.15.100:0/387131895 conn(0x7f46e8077b10 msgr2=0x7f46e8077f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.011+0000 7f46ef1ad700 1 -- 172.21.15.100:0/387131895 shutdown_connections 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.011+0000 7f46ef1ad700 1 -- 172.21.15.100:0/387131895 wait complete. 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ef1ad700 1 Processor -- start 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ef1ad700 1 -- start start 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ef1ad700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e8100b60 0x7f46e81965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ef1ad700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e8101520 0x7f46e8196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ef1ad700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e8106be0 0x7f46e819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ef1ad700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e819b490 con 0x7f46e8100b60 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ef1ad700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e819b600 con 0x7f46e8101520 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ef1ad700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e819b7c0 con 0x7f46e8106be0 2024-02-14T17:44:48.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ecf49700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e8100b60 0x7f46e81965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ecf49700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e8100b60 0x7f46e81965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54226/0 (socket says 172.21.15.100:54226) 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ecf49700 1 -- 172.21.15.100:0/1593944249 learned_addr learned my addr 172.21.15.100:0/1593944249 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ed74a700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e8106be0 0x7f46e819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ecf49700 1 -- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e8106be0 msgr2=0x7f46e819ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ecf49700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e8106be0 0x7f46e819ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46e7fff700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e8101520 0x7f46e8196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ecf49700 1 -- 172.21.15.100:0/1593944249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e8101520 msgr2=0x7f46e8196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ecf49700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e8101520 0x7f46e8196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.012+0000 7f46ecf49700 1 -- 172.21.15.100:0/1593944249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e0009d00 con 0x7f46e8100b60 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.013+0000 7f46ecf49700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e8100b60 0x7f46e81965b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46e000e3c0 tx=0x7f46e000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.013+0000 7f46e5ffb700 1 -- 172.21.15.100:0/1593944249 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0018070 con 0x7f46e8100b60 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.013+0000 7f46e5ffb700 1 -- 172.21.15.100:0/1593944249 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f46e000f040 con 0x7f46e8100b60 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.013+0000 7f46e5ffb700 1 -- 172.21.15.100:0/1593944249 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0004280 con 0x7f46e8100b60 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.013+0000 7f46ef1ad700 1 -- 172.21.15.100:0/1593944249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d8009520 con 0x7f46e8100b60 2024-02-14T17:44:48.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.013+0000 7f46ef1ad700 1 -- 172.21.15.100:0/1593944249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e81a3130 con 0x7f46e8100b60 2024-02-14T17:44:48.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.014+0000 7f46ef1ad700 1 -- 172.21.15.100:0/1593944249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e8068930 con 0x7f46e8100b60 2024-02-14T17:44:48.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.014+0000 7f46e5ffb700 1 -- 172.21.15.100:0/1593944249 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7f46e00090d0 con 0x7f46e8100b60 2024-02-14T17:44:48.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.015+0000 7f46e5ffb700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f46d406d9d0 0x7f46d406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:48.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.015+0000 7f46e7fff700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f46d406d9d0 0x7f46d406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:48.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.015+0000 7f46e5ffb700 1 -- 172.21.15.100:0/1593944249 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f46e008ca20 con 0x7f46e8100b60 2024-02-14T17:44:48.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.015+0000 7f46e7fff700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f46d406d9d0 0x7f46d406fe90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f46e8110d80 tx=0x7f46d0005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:48.021 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.020+0000 7f46e5ffb700 1 -- 172.21.15.100:0/1593944249 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f46e0056760 con 0x7f46e8100b60 2024-02-14T17:44:48.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.216+0000 7f46ef1ad700 1 -- 172.21.15.100:0/1593944249 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46e81100d0 con 0x7f46e8100b60 2024-02-14T17:44:48.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.217+0000 7f46e5ffb700 1 -- 172.21.15.100:0/1593944249 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+13626 (secure 0 0 0) 0x7f46e0059e20 con 0x7f46e8100b60 2024-02-14T17:44:48.219 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:44:48.219 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":58,"fsid":"d69a9998-cb5f-11ee-95b9-87774f69a715","created":"2024-02-14T17:39:23.866791+0000","modified":"2024-02-14T17:44:47.232133+0000","last_up_change":"2024-02-14T17:44:44.144566+0000","last_in_change":"2024-02-14T17:44:27.059067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T17:42:38.342681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d822e357-3e1f-407f-a403-8b994ccc1f84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6803","nonce":2110588638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6805","nonce":2110588638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6809","nonce":2110588638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6807","nonce":2110588638}]},"public_addr":"172.21.15.100:6803/2110588638","cluster_addr":"172.21.15.100:6805/2110588638","heartbeat_back_addr":"172.21.15.100:6809/2110588638","heartbeat_front_addr":"172.21.15.100:6807/2110588638","state":["exists","up"]},{"osd":1,"uuid":"f74efd5d-9766-407c-9b93-8fc21cf42e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6811","nonce":4183142537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6813","nonce":4183142537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6817","nonce":4183142537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6815","nonce":4183142537}]},"public_addr":"172.21.15.100:6811/4183142537","cluster_addr":"172.21.15.100:6813/4183142537","heartbeat_back_addr":"172.21.15.100:6817/4183142537","heartbeat_front_addr":"172.21.15.100:6815/4183142537","state":["exists","up"]},{"osd":2,"uuid":"e476ae41-e2f9-43fa-8cca-7f766e005494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6819","nonce":1857810896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6821","nonce":1857810896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6825","nonce":1857810896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6823","nonce":1857810896}]},"public_addr":"172.21.15.100:6819/1857810896","cluster_addr":"172.21.15.100:6821/1857810896","heartbeat_back_addr":"172.21.15.100:6825/1857810896","heartbeat_front_addr":"172.21.15.100:6823/1857810896","state":["exists","up"]},{"osd":3,"uuid":"0a354e22-5aff-4f94-8764-0b3dfbca48f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6827","nonce":2018852967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6829","nonce":2018852967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6833","nonce":2018852967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6831","nonce":2018852967}]},"public_addr":"172.21.15.100:6827/2018852967","cluster_addr":"172.21.15.100:6829/2018852967","heartbeat_back_addr":"172.21.15.100:6833/2018852967","heartbeat_front_addr":"172.21.15.100:6831/2018852967","state":["exists","up"]},{"osd":4,"uuid":"1a753e42-d9e5-487a-b716-5ff9bd63af60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6801","nonce":952874533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6803","nonce":952874533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6807","nonce":952874533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6805","nonce":952874533}]},"public_addr":"172.21.15.179:6801/952874533","cluster_addr":"172.21.15.179:6803/952874533","heartbeat_back_addr":"172.21.15.179:6807/952874533","heartbeat_front_addr":"172.21.15.179:6805/952874533","state":["exists","up"]},{"osd":5,"uuid":"60fca708-b509-4b49-823b-e9c81ff48e6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6809","nonce":1513852186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6811","nonce":1513852186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6815","nonce":1513852186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6813","nonce":1513852186}]},"public_addr":"172.21.15.179:6809/1513852186","cluster_addr":"172.21.15.179:6811/1513852186","heartbeat_back_addr":"172.21.15.179:6815/1513852186","heartbeat_front_addr":"172.21.15.179:6813/1513852186","state":["exists","up"]},{"osd":6,"uuid":"1d797fe1-a6df-4e3d-a923-8ebbafa7a49b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6817","nonce":2309639257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6819","nonce":2309639257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6823","nonce":2309639257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6821","nonce":2309639257}]},"public_addr":"172.21.15.179:6817/2309639257","cluster_addr":"172.21.15.179:6819/2309639257","heartbeat_back_addr":"172.21.15.179:6823/2309639257","heartbeat_front_addr":"172.21.15.179:6821/2309639257","state":["exists","up"]},{"osd":7,"uuid":"51596ca9-64f6-45fd-87ce-3a83bd7b9c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6825","nonce":1058185483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6827","nonce":1058185483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6831","nonce":1058185483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6829","nonce":1058185483}]},"public_addr":"172.21.15.179:6825/1058185483","cluster_addr":"172.21.15.179:6827/1058185483","heartbeat_back_addr":"172.21.15.179:6831/1058185483","heartbeat_front_addr":"172.21.15.179:6829/1058185483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:41:43.603858+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:42:09.834569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:42:36.005505+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:43:02.929722+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:43:26.403581+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:43:51.680222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:44:17.365479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:44:43.597507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/4153927297":"2024-02-15T17:40:18.013833+0000","172.21.15.100:6800/1559623009":"2024-02-15T17:40:18.013833+0000","172.21.15.100:0/2717619524":"2024-02-15T17:40:18.013833+0000","172.21.15.100:0/3583073370":"2024-02-15T17:39:51.038137+0000","172.21.15.100:6801/1559623009":"2024-02-15T17:40:18.013833+0000","172.21.15.100:0/2663948233":"2024-02-15T17:39:51.038137+0000","172.21.15.100:0/2115528537":"2024-02-15T17:39:51.038137+0000","172.21.15.100:6800/2036660321":"2024-02-15T17:39:51.038137+0000","172.21.15.100:6801/2036660321":"2024-02-15T17:39:51.038137+0000","172.21.15.100:0/3950545425":"2024-02-15T17:40:18.013833+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T17:44:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 -- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f46d406d9d0 msgr2=0x7f46d406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f46d406d9d0 0x7f46d406fe90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f46e8110d80 tx=0x7f46d0005cf0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 -- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e8100b60 msgr2=0x7f46e81965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e8100b60 0x7f46e81965b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46e000e3c0 tx=0x7f46e000e6d0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 -- 172.21.15.100:0/1593944249 shutdown_connections 2024-02-14T17:44:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f46e8106be0 0x7f46e819ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46e8100b60 0x7f46e81965b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f46d406d9d0 0x7f46d406fe90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 --2- 172.21.15.100:0/1593944249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46e8101520 0x7f46e8196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:48.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 -- 172.21.15.100:0/1593944249 >> 172.21.15.100:0/1593944249 conn(0x7f46e8077b10 msgr2=0x7f46e81046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:48.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 -- 172.21.15.100:0/1593944249 shutdown_connections 2024-02-14T17:44:48.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:48.219+0000 7f46ef1ad700 1 -- 172.21.15.100:0/1593944249 wait complete. 2024-02-14T17:44:48.339 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:48 smithi179 ceph-mon[33633]: osdmap e58: 8 total, 8 up, 8 in 2024-02-14T17:44:48.339 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:48 smithi179 ceph-mon[33633]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:48.339 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1593944249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:44:48.515 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:48 smithi100 ceph-mon[39557]: osdmap e58: 8 total, 8 up, 8 in 2024-02-14T17:44:48.515 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:48 smithi100 ceph-mon[39557]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:48.515 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1593944249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:44:48.515 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:48 smithi100 ceph-mon[30138]: osdmap e58: 8 total, 8 up, 8 in 2024-02-14T17:44:48.515 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:48 smithi100 ceph-mon[30138]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:48.515 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1593944249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:44:48.854 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-14T17:42:38.342681+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-14T17:44:48.854 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-14T17:44:49.090 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:44:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.654+0000 7f6e5d141700 1 -- 172.21.15.100:0/1289138069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58101810 msgr2=0x7f6e5810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.654+0000 7f6e5d141700 1 --2- 172.21.15.100:0/1289138069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58101810 0x7f6e5810da40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009f90 tx=0x7f6e5000b330 comp rx=0 tx=0).stop 2024-02-14T17:44:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.654+0000 7f6e5d141700 1 -- 172.21.15.100:0/1289138069 shutdown_connections 2024-02-14T17:44:50.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.654+0000 7f6e5d141700 1 --2- 172.21.15.100:0/1289138069 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58101810 0x7f6e5810da40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.654+0000 7f6e5d141700 1 --2- 172.21.15.100:0/1289138069 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e58106d60 0x7f6e58107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.654+0000 7f6e5d141700 1 --2- 172.21.15.100:0/1289138069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e58100e50 0x7f6e581012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.654+0000 7f6e5d141700 1 -- 172.21.15.100:0/1289138069 >> 172.21.15.100:0/1289138069 conn(0x7f6e580779d0 msgr2=0x7f6e58077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.654+0000 7f6e5d141700 1 -- 172.21.15.100:0/1289138069 shutdown_connections 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.655+0000 7f6e5d141700 1 -- 172.21.15.100:0/1289138069 wait complete. 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.655+0000 7f6e5d141700 1 Processor -- start 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.655+0000 7f6e5d141700 1 -- start start 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e5d141700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58100e50 0x7f6e5819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e5d141700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e58101810 0x7f6e5819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e5d141700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e58106d60 0x7f6e58197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e5d141700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58197e70 con 0x7f6e58100e50 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e5d141700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58197fe0 con 0x7f6e58101810 2024-02-14T17:44:50.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e5d141700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581981a0 con 0x7f6e58106d60 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e56d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58100e50 0x7f6e5819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e56d9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58100e50 0x7f6e5819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54234/0 (socket says 172.21.15.100:54234) 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e56d9d700 1 -- 172.21.15.100:0/3535935226 learned_addr learned my addr 172.21.15.100:0/3535935226 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e5759e700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e58106d60 0x7f6e58197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e5659c700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e58101810 0x7f6e5819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e56d9d700 1 -- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e58106d60 msgr2=0x7f6e58197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e56d9d700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e58106d60 0x7f6e58197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e56d9d700 1 -- 172.21.15.100:0/3535935226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e58101810 msgr2=0x7f6e5819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e56d9d700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e58101810 0x7f6e5819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e56d9d700 1 -- 172.21.15.100:0/3535935226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480095f0 con 0x7f6e58100e50 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e56d9d700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58100e50 0x7f6e5819d6a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800ec10 tx=0x7f6e4800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e3ffff700 1 -- 172.21.15.100:0/3535935226 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800cc90 con 0x7f6e58100e50 2024-02-14T17:44:50.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e3ffff700 1 -- 172.21.15.100:0/3535935226 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6e4800cdf0 con 0x7f6e58100e50 2024-02-14T17:44:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e3ffff700 1 -- 172.21.15.100:0/3535935226 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e48003df0 con 0x7f6e58100e50 2024-02-14T17:44:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e5d141700 1 -- 172.21.15.100:0/3535935226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e50009c40 con 0x7f6e58100e50 2024-02-14T17:44:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.656+0000 7f6e5d141700 1 -- 172.21.15.100:0/3535935226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e581a44c0 con 0x7f6e58100e50 2024-02-14T17:44:50.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.658+0000 7f6e5d141700 1 -- 172.21.15.100:0/3535935226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e581022f0 con 0x7f6e58100e50 2024-02-14T17:44:50.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.658+0000 7f6e3ffff700 1 -- 172.21.15.100:0/3535935226 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7f6e48003f90 con 0x7f6e58100e50 2024-02-14T17:44:50.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.658+0000 7f6e3ffff700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6e4006d9d0 0x7f6e4006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:50.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.658+0000 7f6e5659c700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6e4006d9d0 0x7f6e4006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:50.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.659+0000 7f6e3ffff700 1 -- 172.21.15.100:0/3535935226 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f6e48014070 con 0x7f6e58100e50 2024-02-14T17:44:50.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.659+0000 7f6e5659c700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6e4006d9d0 0x7f6e4006fe90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e58198c70 tx=0x7f6e4c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:50.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.663+0000 7f6e3ffff700 1 -- 172.21.15.100:0/3535935226 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e48056fc0 con 0x7f6e58100e50 2024-02-14T17:44:50.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.853+0000 7f6e5d141700 1 -- 172.21.15.100:0/3535935226 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e58198d10 con 0x7f6e58100e50 2024-02-14T17:44:50.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.854+0000 7f6e3ffff700 1 -- 172.21.15.100:0/3535935226 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e4805a680 con 0x7f6e58100e50 2024-02-14T17:44:50.856 INFO:teuthology.orchestra.run.smithi100.stdout:pg_num: 1 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 -- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6e4006d9d0 msgr2=0x7f6e4006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6e4006d9d0 0x7f6e4006fe90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e58198c70 tx=0x7f6e4c009190 comp rx=0 tx=0).stop 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 -- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58100e50 msgr2=0x7f6e5819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58100e50 0x7f6e5819d6a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800ec10 tx=0x7f6e4800ef20 comp rx=0 tx=0).stop 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 -- 172.21.15.100:0/3535935226 shutdown_connections 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6e58106d60 0x7f6e58197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e58100e50 0x7f6e5819d6a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6e4006d9d0 0x7f6e4006fe90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 --2- 172.21.15.100:0/3535935226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e58101810 0x7f6e5819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 -- 172.21.15.100:0/3535935226 >> 172.21.15.100:0/3535935226 conn(0x7f6e580779d0 msgr2=0x7f6e580ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 -- 172.21.15.100:0/3535935226 shutdown_connections 2024-02-14T17:44:50.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:44:50.857+0000 7f6e5d141700 1 -- 172.21.15.100:0/3535935226 wait complete. 2024-02-14T17:44:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:51 smithi100 ceph-mon[30138]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-02-14T17:44:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3535935226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T17:44:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:51 smithi100 ceph-mon[39557]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-02-14T17:44:51.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3535935226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T17:44:51.507 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi100 2024-02-14T17:44:51.508 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi100=foo.a' 2024-02-14T17:44:51.539 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:51 smithi179 ceph-mon[33633]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-02-14T17:44:51.540 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3535935226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-14T17:44:51.759 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:44:53.314 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:53 smithi179 ceph-mon[33633]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-14T17:44:53.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:53 smithi100 ceph-mon[30138]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-14T17:44:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:53 smithi100 ceph-mon[39557]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-02-14T17:44:53.418 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.415+0000 7f4cb5943700 1 -- 172.21.15.179:0/3899574166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cb0102770 msgr2=0x7f4cb0109020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.415+0000 7f4cb5943700 1 --2- 172.21.15.179:0/3899574166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cb0102770 0x7f4cb0109020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ca800b0d0 tx=0x7f4ca800b3e0 comp rx=0 tx=0).stop 2024-02-14T17:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.415+0000 7f4cb5943700 1 -- 172.21.15.179:0/3899574166 shutdown_connections 2024-02-14T17:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.415+0000 7f4cb5943700 1 --2- 172.21.15.179:0/3899574166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cb0102770 0x7f4cb0109020 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.415+0000 7f4cb5943700 1 --2- 172.21.15.179:0/3899574166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb0100bd0 0x7f4cb0100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.416+0000 7f4cb5943700 1 --2- 172.21.15.179:0/3899574166 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cb0101dd0 0x7f4cb0102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.416+0000 7f4cb5943700 1 -- 172.21.15.179:0/3899574166 >> 172.21.15.179:0/3899574166 conn(0x7f4cb00fc170 msgr2=0x7f4cb00fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:53.419 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.416+0000 7f4cb5943700 1 -- 172.21.15.179:0/3899574166 shutdown_connections 2024-02-14T17:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.416+0000 7f4cb5943700 1 -- 172.21.15.179:0/3899574166 wait complete. 2024-02-14T17:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.416+0000 7f4cb5943700 1 Processor -- start 2024-02-14T17:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.416+0000 7f4cb5943700 1 -- start start 2024-02-14T17:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.416+0000 7f4cb5943700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cb0100bd0 0x7f4cb0104b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.416+0000 7f4cb5943700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb0101dd0 0x7f4cb01031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:53.420 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.416+0000 7f4cb5943700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cb0102770 0x7f4cb01036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4cb5943700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb0103e70 con 0x7f4cb0101dd0 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4cb5943700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb0103fe0 con 0x7f4cb0102770 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4cb5943700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb01a12f0 con 0x7f4cb0100bd0 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4caf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cb0102770 0x7f4cb01036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4caf7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cb0102770 0x7f4cb01036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55770/0 (socket says 172.21.15.179:55770) 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4caf7fe700 1 -- 172.21.15.179:0/73385605 learned_addr learned my addr 172.21.15.179:0/73385605 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4caeffd700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cb0100bd0 0x7f4cb0104b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4cae7fc700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb0101dd0 0x7f4cb01031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4caf7fe700 1 -- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cb0100bd0 msgr2=0x7f4cb0104b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4caf7fe700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cb0100bd0 0x7f4cb0104b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4caf7fe700 1 -- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb0101dd0 msgr2=0x7f4cb01031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4caf7fe700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb0101dd0 0x7f4cb01031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4caf7fe700 1 -- 172.21.15.179:0/73385605 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca8009d00 con 0x7f4cb0102770 2024-02-14T17:44:53.421 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4caf7fe700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cb0102770 0x7f4cb01036f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009800 tx=0x7f4ca8005b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4cb4941700 1 -- 172.21.15.179:0/73385605 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca800e040 con 0x7f4cb0102770 2024-02-14T17:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4cb4941700 1 -- 172.21.15.179:0/73385605 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f4ca8009460 con 0x7f4cb0102770 2024-02-14T17:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4cb5943700 1 -- 172.21.15.179:0/73385605 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb01a1570 con 0x7f4cb0102770 2024-02-14T17:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4cb4941700 1 -- 172.21.15.179:0/73385605 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca801b6d0 con 0x7f4cb0102770 2024-02-14T17:44:53.422 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.417+0000 7f4cb5943700 1 -- 172.21.15.179:0/73385605 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb01a1980 con 0x7f4cb0102770 2024-02-14T17:44:53.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.418+0000 7f4cb5943700 1 -- 172.21.15.179:0/73385605 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb01089b0 con 0x7f4cb0102770 2024-02-14T17:44:53.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.419+0000 7f4cb4941700 1 -- 172.21.15.179:0/73385605 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7f4ca8005070 con 0x7f4cb0102770 2024-02-14T17:44:53.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.419+0000 7f4cb4941700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f4c9806d9d0 0x7f4c9806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:53.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.419+0000 7f4cb4941700 1 -- 172.21.15.179:0/73385605 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f4ca808e230 con 0x7f4cb0102770 2024-02-14T17:44:53.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.419+0000 7f4caeffd700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f4c9806d9d0 0x7f4c9806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:53.423 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.420+0000 7f4caeffd700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f4c9806d9d0 0x7f4c9806fe90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0101c30 tx=0x7f4ca000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:53.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.424+0000 7f4cb4941700 1 -- 172.21.15.179:0/73385605 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ca8057ef0 con 0x7f4cb0102770 2024-02-14T17:44:53.632 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.631+0000 7f4cb5943700 1 -- 172.21.15.179:0/73385605 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi100=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cb00620b0 con 0x7f4c9806d9d0 2024-02-14T17:44:53.640 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.639+0000 7f4cb4941700 1 -- 172.21.15.179:0/73385605 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f4cb00620b0 con 0x7f4c9806d9d0 2024-02-14T17:44:53.640 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled rgw.foo.a update... 2024-02-14T17:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 -- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f4c9806d9d0 msgr2=0x7f4c9806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f4c9806d9d0 0x7f4c9806fe90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0101c30 tx=0x7f4ca000b410 comp rx=0 tx=0).stop 2024-02-14T17:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 -- 172.21.15.179:0/73385605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cb0102770 msgr2=0x7f4cb01036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cb0102770 0x7f4cb01036f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009800 tx=0x7f4ca8005b80 comp rx=0 tx=0).stop 2024-02-14T17:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 -- 172.21.15.179:0/73385605 shutdown_connections 2024-02-14T17:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f4c9806d9d0 0x7f4c9806fe90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cb0102770 0x7f4cb01036f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4cb0100bd0 0x7f4cb0104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 --2- 172.21.15.179:0/73385605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4cb0101dd0 0x7f4cb01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:53.642 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 -- 172.21.15.179:0/73385605 >> 172.21.15.179:0/73385605 conn(0x7f4cb00fc170 msgr2=0x7f4cb006aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:53.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 -- 172.21.15.179:0/73385605 shutdown_connections 2024-02-14T17:44:53.643 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:53.641+0000 7f4cb5943700 1 -- 172.21.15.179:0/73385605 wait complete. 2024-02-14T17:44:54.248 DEBUG:teuthology.orchestra.run.smithi100:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@rgw.foo.a.service 2024-02-14T17:44:54.250 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi179 2024-02-14T17:44:54.250 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-02-14T17:44:54.522 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:44:54.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi100=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi100=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:54 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:54.907 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi100=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:44:54.907 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-02-14T17:44:54.907 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:54.908 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:54.908 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-14T17:44:54.908 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:54.908 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:44:54.908 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:54.908 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-14T17:44:54.908 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-14T17:44:54.908 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:54.908 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:54 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:44:55.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:55 smithi179 ceph-mon[33633]: Deploying daemon rgw.foo.a on smithi100 2024-02-14T17:44:55.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:55 smithi100 ceph-mon[30138]: Deploying daemon rgw.foo.a on smithi100 2024-02-14T17:44:55.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:55 smithi100 ceph-mon[39557]: Deploying daemon rgw.foo.a on smithi100 2024-02-14T17:44:56.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.069+0000 7fcb54536700 1 -- 172.21.15.179:0/90685431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb4c101d90 msgr2=0x7fcb4c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:56.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.069+0000 7fcb54536700 1 --2- 172.21.15.179:0/90685431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb4c101d90 0x7fcb4c1021f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb480099c0 tx=0x7fcb48009cd0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.069+0000 7fcb54536700 1 -- 172.21.15.179:0/90685431 shutdown_connections 2024-02-14T17:44:56.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.069+0000 7fcb54536700 1 --2- 172.21.15.179:0/90685431 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb4c102730 0x7fcb4c108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.069+0000 7fcb54536700 1 --2- 172.21.15.179:0/90685431 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4c100b90 0x7fcb4c100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.069+0000 7fcb54536700 1 --2- 172.21.15.179:0/90685431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb4c101d90 0x7fcb4c1021f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.069+0000 7fcb54536700 1 -- 172.21.15.179:0/90685431 >> 172.21.15.179:0/90685431 conn(0x7fcb4c0fc130 msgr2=0x7fcb4c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:56.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.069+0000 7fcb54536700 1 -- 172.21.15.179:0/90685431 shutdown_connections 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.069+0000 7fcb54536700 1 -- 172.21.15.179:0/90685431 wait complete. 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb54536700 1 Processor -- start 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb54536700 1 -- start start 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb54536700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4c100b90 0x7fcb4c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb54536700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb4c101d90 0x7fcb4c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb54536700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb4c102730 0x7fcb4c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb54536700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c19ed60 con 0x7fcb4c101d90 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb54536700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c19eea0 con 0x7fcb4c102730 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb54536700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c19f030 con 0x7fcb4c100b90 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb52ad3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb4c102730 0x7fcb4c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb522d2700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4c100b90 0x7fcb4c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb52ad3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb4c102730 0x7fcb4c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55794/0 (socket says 172.21.15.179:55794) 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb522d2700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4c100b90 0x7fcb4c1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.179:49692/0 (socket says 172.21.15.179:49692) 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb52ad3700 1 -- 172.21.15.179:0/3336212780 learned_addr learned my addr 172.21.15.179:0/3336212780 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:44:56.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb51ad1700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb4c101d90 0x7fcb4c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb52ad3700 1 -- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4c100b90 msgr2=0x7fcb4c1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb52ad3700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4c100b90 0x7fcb4c1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb52ad3700 1 -- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb4c101d90 msgr2=0x7fcb4c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb52ad3700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb4c101d90 0x7fcb4c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.070+0000 7fcb52ad3700 1 -- 172.21.15.179:0/3336212780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480095f0 con 0x7fcb4c102730 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.071+0000 7fcb52ad3700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb4c102730 0x7fcb4c19e790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb44007ee0 tx=0x7fcb4400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.071+0000 7fcb3f7fe700 1 -- 172.21.15.179:0/3336212780 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400dd20 con 0x7fcb4c102730 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.071+0000 7fcb3f7fe700 1 -- 172.21.15.179:0/3336212780 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcb4400f040 con 0x7fcb4c102730 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.071+0000 7fcb3f7fe700 1 -- 172.21.15.179:0/3336212780 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb440046c0 con 0x7fcb4c102730 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.071+0000 7fcb54536700 1 -- 172.21.15.179:0/3336212780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4c19f310 con 0x7fcb4c102730 2024-02-14T17:44:56.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.071+0000 7fcb54536700 1 -- 172.21.15.179:0/3336212780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c19f810 con 0x7fcb4c102730 2024-02-14T17:44:56.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.072+0000 7fcb54536700 1 -- 172.21.15.179:0/3336212780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4c068930 con 0x7fcb4c102730 2024-02-14T17:44:56.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.072+0000 7fcb3f7fe700 1 -- 172.21.15.179:0/3336212780 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7fcb44003a00 con 0x7fcb4c102730 2024-02-14T17:44:56.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.072+0000 7fcb3f7fe700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcb3806d9d0 0x7fcb3806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:44:56.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.073+0000 7fcb3f7fe700 1 -- 172.21.15.179:0/3336212780 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5287+0+0 (secure 0 0 0) 0x7fcb4408c740 con 0x7fcb4c102730 2024-02-14T17:44:56.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.073+0000 7fcb522d2700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcb3806d9d0 0x7fcb3806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:44:56.077 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.073+0000 7fcb522d2700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcb3806d9d0 0x7fcb3806fe90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c101bf0 tx=0x7fcb4000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:44:56.079 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.077+0000 7fcb3f7fe700 1 -- 172.21.15.179:0/3336212780 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb44056480 con 0x7fcb4c102730 2024-02-14T17:44:56.272 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.270+0000 7fcb54536700 1 -- 172.21.15.179:0/3336212780 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7fcb4c10bb20 con 0x7fcb4c102730 2024-02-14T17:44:56.651 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 17:44:56 smithi100 systemd[1]: Starting Ceph rgw.foo.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:44:56.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.655+0000 7fcb3f7fe700 1 -- 172.21.15.179:0/3336212780 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v59) v1 ==== 160+0+0 (secure 0 0 0) 0x7fcb44059b40 con 0x7fcb4c102730 2024-02-14T17:44:56.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 -- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcb3806d9d0 msgr2=0x7fcb3806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcb3806d9d0 0x7fcb3806fe90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c101bf0 tx=0x7fcb4000b410 comp rx=0 tx=0).stop 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 -- 172.21.15.179:0/3336212780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb4c102730 msgr2=0x7fcb4c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb4c102730 0x7fcb4c19e790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb44007ee0 tx=0x7fcb4400d3b0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 -- 172.21.15.179:0/3336212780 shutdown_connections 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fcb3806d9d0 0x7fcb3806fe90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb4c102730 0x7fcb4c19e790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4c100b90 0x7fcb4c1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 --2- 172.21.15.179:0/3336212780 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb4c101d90 0x7fcb4c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 -- 172.21.15.179:0/3336212780 >> 172.21.15.179:0/3336212780 conn(0x7fcb4c0fc130 msgr2=0x7fcb4c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 -- 172.21.15.179:0/3336212780 shutdown_connections 2024-02-14T17:44:56.659 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:44:56.657+0000 7fcb54536700 1 -- 172.21.15.179:0/3336212780 wait complete. 2024-02-14T17:44:56.660 INFO:teuthology.orchestra.run.smithi179.stderr:pool 'datapool' created 2024-02-14T17:44:56.977 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:56 smithi179 ceph-mon[33633]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-14T17:44:56.977 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/3336212780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-14T17:44:56.978 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-14T17:44:56.985 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:56 smithi100 ceph-mon[39557]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-14T17:44:56.985 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/3336212780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-14T17:44:56.985 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-14T17:44:56.985 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:56 smithi100 ceph-mon[30138]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-14T17:44:56.985 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/3336212780' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-14T17:44:56.985 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-14T17:44:56.986 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 17:44:56 smithi100 podman[67756]: 2024-02-14 17:44:56.65659842 +0000 UTC m=+0.103805002 container create 63622df17c34d3d9d3f8b0de18dbad67d1e4ac9e60324c05f84404b24b7b2007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-14T17:44:56.986 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 17:44:56 smithi100 podman[67756]: 2024-02-14 17:44:56.570631496 +0000 UTC m=+0.017838095 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:44:56.986 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 17:44:56 smithi100 podman[67756]: 2024-02-14 17:44:56.884420361 +0000 UTC m=+0.331626946 container init 63622df17c34d3d9d3f8b0de18dbad67d1e4ac9e60324c05f84404b24b7b2007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T17:44:56.986 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 17:44:56 smithi100 podman[67756]: 2024-02-14 17:44:56.889517942 +0000 UTC m=+0.336724525 container start 63622df17c34d3d9d3f8b0de18dbad67d1e4ac9e60324c05f84404b24b7b2007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-14T17:44:56.986 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 17:44:56 smithi100 bash[67756]: 63622df17c34d3d9d3f8b0de18dbad67d1e4ac9e60324c05f84404b24b7b2007 2024-02-14T17:44:57.336 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- rbd pool init datapool 2024-02-14T17:44:57.400 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 17:44:56 smithi100 systemd[1]: Started Ceph rgw.foo.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:44:57.573 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:44:57.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-14T17:44:57.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:57 smithi179 ceph-mon[33633]: osdmap e59: 8 total, 8 up, 8 in 2024-02-14T17:44:57.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:57 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:57.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:57 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:57.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:57 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:57.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:57 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:57.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:57 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:57.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:57 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:58.104 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-14T17:44:58.105 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[30138]: osdmap e59: 8 total, 8 up, 8 in 2024-02-14T17:44:58.105 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.105 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.105 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.105 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.105 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.105 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:58.105 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-14T17:44:58.105 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[39557]: osdmap e59: 8 total, 8 up, 8 in 2024-02-14T17:44:58.105 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.106 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.106 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.106 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.106 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.106 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:57 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:44:58.678 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:58 smithi179 ceph-mon[33633]: Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-02-14T17:44:58.678 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:58 smithi179 ceph-mon[33633]: osdmap e60: 8 total, 8 up, 8 in 2024-02-14T17:44:58.678 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1152962022' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:44:58.678 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:44:58.678 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:58 smithi179 ceph-mon[33633]: pgmap v190: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:58.679 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:58 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:58.679 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-14T17:44:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[39557]: Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-02-14T17:44:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[39557]: osdmap e60: 8 total, 8 up, 8 in 2024-02-14T17:44:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1152962022' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:44:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:44:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[39557]: pgmap v190: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:59.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:59.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-14T17:44:59.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[30138]: Saving service rgw.foo.a spec with placement smithi100=foo.a;count:1 2024-02-14T17:44:59.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[30138]: osdmap e60: 8 total, 8 up, 8 in 2024-02-14T17:44:59.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1152962022' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:44:59.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-14T17:44:59.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[30138]: pgmap v190: 36 pgs: 35 unknown, 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:44:59.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:44:59.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-14T17:45:00.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: osdmap e61: 8 total, 8 up, 8 in 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: Checking dashboard <-> RGW credentials 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/3041391103' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: osdmap e62: 8 total, 8 up, 8 in 2024-02-14T17:45:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:44:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: osdmap e61: 8 total, 8 up, 8 in 2024-02-14T17:45:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: Checking dashboard <-> RGW credentials 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/3041391103' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: osdmap e62: 8 total, 8 up, 8 in 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: osdmap e61: 8 total, 8 up, 8 in 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:00.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:45:00.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: Checking dashboard <-> RGW credentials 2024-02-14T17:45:00.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/3041391103' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:45:00.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-14T17:45:00.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-14T17:45:00.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: osdmap e62: 8 total, 8 up, 8 in 2024-02-14T17:45:00.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:44:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:00 smithi179 ceph-mon[33633]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:00 smithi179 ceph-mon[33633]: pgmap v193: 68 pgs: 15 active+clean, 4 creating+peering, 49 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:45:01.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:45:01.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:00 smithi179 ceph-mon[33633]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:45:01.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:00 smithi179 ceph-mon[33633]: osdmap e63: 8 total, 8 up, 8 in 2024-02-14T17:45:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[39557]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[39557]: pgmap v193: 68 pgs: 15 active+clean, 4 creating+peering, 49 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:45:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:45:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[39557]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:45:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[39557]: osdmap e63: 8 total, 8 up, 8 in 2024-02-14T17:45:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[30138]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-14T17:45:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[30138]: pgmap v193: 68 pgs: 15 active+clean, 4 creating+peering, 49 unknown; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:45:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:45:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[30138]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-14T17:45:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:00 smithi100 ceph-mon[30138]: osdmap e63: 8 total, 8 up, 8 in 2024-02-14T17:45:02.270 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.179 --placement '1;smithi179=iscsi.a' 2024-02-14T17:45:02.512 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:45:02.974 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:02 smithi179 ceph-mon[33633]: osdmap e64: 8 total, 8 up, 8 in 2024-02-14T17:45:02.974 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:02.974 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:02 smithi179 ceph-mon[33633]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:02.975 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:02.975 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:02.975 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:02 smithi179 ceph-mon[33633]: pgmap v196: 100 pgs: 34 active+clean, 14 creating+peering, 52 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-14T17:45:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[39557]: osdmap e64: 8 total, 8 up, 8 in 2024-02-14T17:45:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[39557]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[39557]: pgmap v196: 100 pgs: 34 active+clean, 14 creating+peering, 52 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-14T17:45:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[30138]: osdmap e64: 8 total, 8 up, 8 in 2024-02-14T17:45:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[30138]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-14T17:45:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:02 smithi100 ceph-mon[30138]: pgmap v196: 100 pgs: 34 active+clean, 14 creating+peering, 52 unknown; 578 KiB data, 217 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-14T17:45:03.996 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:03 smithi179 ceph-mon[33633]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:45:03.996 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:45:03.996 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:03 smithi179 ceph-mon[33633]: osdmap e65: 8 total, 8 up, 8 in 2024-02-14T17:45:04.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:03 smithi100 ceph-mon[30138]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:45:04.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:45:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:03 smithi100 ceph-mon[30138]: osdmap e65: 8 total, 8 up, 8 in 2024-02-14T17:45:04.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:03 smithi100 ceph-mon[39557]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:45:04.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-14T17:45:04.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:03 smithi100 ceph-mon[39557]: osdmap e65: 8 total, 8 up, 8 in 2024-02-14T17:45:04.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.252+0000 7fed1fddc700 1 -- 172.21.15.179:0/504040222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed181058b0 msgr2=0x7fed18105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:04.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 --2- 172.21.15.179:0/504040222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed181058b0 0x7fed18105d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed1000b0a0 tx=0x7fed1000b3b0 comp rx=0 tx=0).stop 2024-02-14T17:45:04.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 -- 172.21.15.179:0/504040222 shutdown_connections 2024-02-14T17:45:04.256 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 --2- 172.21.15.179:0/504040222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed181058b0 0x7fed18105d30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 --2- 172.21.15.179:0/504040222 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed18100570 0x7fed181029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 --2- 172.21.15.179:0/504040222 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed18102ee0 0x7fed18105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 -- 172.21.15.179:0/504040222 >> 172.21.15.179:0/504040222 conn(0x7fed180fc190 msgr2=0x7fed180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 -- 172.21.15.179:0/504040222 shutdown_connections 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 -- 172.21.15.179:0/504040222 wait complete. 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 Processor -- start 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 -- start start 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed18100570 0x7fed1819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed18102ee0 0x7fed1819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed181058b0 0x7fed181a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed181a5350 con 0x7fed18100570 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed181a5490 con 0x7fed18102ee0 2024-02-14T17:45:04.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.253+0000 7fed1fddc700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed181a5620 con 0x7fed181058b0 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1d377700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed18102ee0 0x7fed1819dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1d377700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed18102ee0 0x7fed1819dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56760/0 (socket says 172.21.15.179:56760) 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1d377700 1 -- 172.21.15.179:0/345347923 learned_addr learned my addr 172.21.15.179:0/345347923 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1db78700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed18100570 0x7fed1819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1e379700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed181058b0 0x7fed181a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1d377700 1 -- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed181058b0 msgr2=0x7fed181a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1d377700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed181058b0 0x7fed181a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1d377700 1 -- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed18100570 msgr2=0x7fed1819d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1d377700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed18100570 0x7fed1819d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1d377700 1 -- 172.21.15.179:0/345347923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed10009d00 con 0x7fed18102ee0 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1d377700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed18102ee0 0x7fed1819dd00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed0800d950 tx=0x7fed0800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed0effd700 1 -- 172.21.15.179:0/345347923 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0800f890 con 0x7fed18102ee0 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed0effd700 1 -- 172.21.15.179:0/345347923 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fed0800fe70 con 0x7fed18102ee0 2024-02-14T17:45:04.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed0effd700 1 -- 172.21.15.179:0/345347923 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0800e3b0 con 0x7fed18102ee0 2024-02-14T17:45:04.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1fddc700 1 -- 172.21.15.179:0/345347923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed181a5900 con 0x7fed18102ee0 2024-02-14T17:45:04.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.254+0000 7fed1fddc700 1 -- 172.21.15.179:0/345347923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed181a5db0 con 0x7fed18102ee0 2024-02-14T17:45:04.259 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.256+0000 7fed1fddc700 1 -- 172.21.15.179:0/345347923 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed180fdd10 con 0x7fed18102ee0 2024-02-14T17:45:04.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.256+0000 7fed0effd700 1 -- 172.21.15.179:0/345347923 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7fed0800f9f0 con 0x7fed18102ee0 2024-02-14T17:45:04.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.256+0000 7fed0effd700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed0406da70 0x7fed0406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:04.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.256+0000 7fed0effd700 1 -- 172.21.15.179:0/345347923 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7121+0+0 (secure 0 0 0) 0x7fed0808d350 con 0x7fed18102ee0 2024-02-14T17:45:04.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.256+0000 7fed1db78700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed0406da70 0x7fed0406ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:04.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.257+0000 7fed1db78700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed0406da70 0x7fed0406ff30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed14005950 tx=0x7fed1400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:04.263 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.261+0000 7fed0effd700 1 -- 172.21.15.179:0/345347923 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed080568e0 con 0x7fed18102ee0 2024-02-14T17:45:04.466 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.465+0000 7fed1fddc700 1 -- 172.21.15.179:0/345347923 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.179", "placement": "1;smithi179=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fed180620b0 con 0x7fed0406da70 2024-02-14T17:45:04.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.473+0000 7fed0effd700 1 -- 172.21.15.179:0/345347923 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7fed180620b0 con 0x7fed0406da70 2024-02-14T17:45:04.475 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled iscsi.datapool update... 2024-02-14T17:45:04.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 -- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed0406da70 msgr2=0x7fed0406ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:04.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed0406da70 0x7fed0406ff30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fed14005950 tx=0x7fed1400b410 comp rx=0 tx=0).stop 2024-02-14T17:45:04.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 -- 172.21.15.179:0/345347923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed18102ee0 msgr2=0x7fed1819dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:04.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed18102ee0 0x7fed1819dd00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fed0800d950 tx=0x7fed0800dc60 comp rx=0 tx=0).stop 2024-02-14T17:45:04.477 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 -- 172.21.15.179:0/345347923 shutdown_connections 2024-02-14T17:45:04.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed181058b0 0x7fed181a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:04.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed18100570 0x7fed1819d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:04.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed0406da70 0x7fed0406ff30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:04.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 --2- 172.21.15.179:0/345347923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed18102ee0 0x7fed1819dd00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:04.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 -- 172.21.15.179:0/345347923 >> 172.21.15.179:0/345347923 conn(0x7fed180fc190 msgr2=0x7fed18103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:04.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 -- 172.21.15.179:0/345347923 shutdown_connections 2024-02-14T17:45:04.478 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:04.475+0000 7fed1fddc700 1 -- 172.21.15.179:0/345347923 wait complete. 2024-02-14T17:45:05.056 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:04 smithi179 ceph-mon[33633]: osdmap e66: 8 total, 8 up, 8 in 2024-02-14T17:45:05.056 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.056 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.056 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.056 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:04 smithi179 ceph-mon[33633]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.056 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:04 smithi179 ceph-mon[33633]: pgmap v199: 132 pgs: 72 active+clean, 20 creating+peering, 40 unknown; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:45:05.056 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:04 smithi179 ceph-mon[33633]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:05.056 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:04 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:05.077 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-02-14T17:45:05.078 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:45:05.078 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-14T17:45:05.108 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[39557]: osdmap e66: 8 total, 8 up, 8 in 2024-02-14T17:45:05.197 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[39557]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[39557]: pgmap v199: 132 pgs: 72 active+clean, 20 creating+peering, 40 unknown; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:45:05.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[39557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:05.198 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:05.198 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[30138]: osdmap e66: 8 total, 8 up, 8 in 2024-02-14T17:45:05.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[30138]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-14T17:45:05.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[30138]: pgmap v199: 132 pgs: 72 active+clean, 20 creating+peering, 40 unknown; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:45:05.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[30138]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:45:05.199 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:04 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:05.200 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:45:05.200 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-14T17:45:05.229 DEBUG:teuthology.orchestra.run.smithi179:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@iscsi.iscsi.a.service 2024-02-14T17:45:05.273 INFO:tasks.cephadm:Adding prometheus.a on smithi179 2024-02-14T17:45:05.273 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply prometheus '1;smithi179=a' 2024-02-14T17:45:05.544 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:45:05.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:05 smithi179 ceph-mon[33633]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.179", "placement": "1;smithi179=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:05.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:05 smithi179 ceph-mon[33633]: Saving service iscsi.datapool spec with placement smithi179=iscsi.a;count:1 2024-02-14T17:45:05.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:45:05.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:05 smithi179 ceph-mon[33633]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:45:05.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:05 smithi179 ceph-mon[33633]: osdmap e67: 8 total, 8 up, 8 in 2024-02-14T17:45:05.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:05.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:05.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:05 smithi179 ceph-mon[33633]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:05.924 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[30138]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.179", "placement": "1;smithi179=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[30138]: Saving service iscsi.datapool spec with placement smithi179=iscsi.a;count:1 2024-02-14T17:45:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:45:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[30138]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:45:06.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[30138]: osdmap e67: 8 total, 8 up, 8 in 2024-02-14T17:45:06.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:06.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:06.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[30138]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:06.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[39557]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.179", "placement": "1;smithi179=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[39557]: Saving service iscsi.datapool spec with placement smithi179=iscsi.a;count:1 2024-02-14T17:45:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:45:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[39557]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-14T17:45:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[39557]: osdmap e67: 8 total, 8 up, 8 in 2024-02-14T17:45:06.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3724405212' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:06.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3822219600' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:06.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[39557]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:06.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-14T17:45:06.945 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:45:06.945 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:45:06.945 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: osdmap e68: 8 total, 8 up, 8 in 2024-02-14T17:45:06.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:06.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:06.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:06.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-14T17:45:06.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-14T17:45:06.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:06.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: Deploying daemon iscsi.iscsi.a on smithi179 2024-02-14T17:45:06.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:06 smithi179 ceph-mon[33633]: pgmap v202: 132 pgs: 100 active+clean, 16 creating+peering, 16 unknown; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:45:07.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.129+0000 7f62f11a5700 1 -- 172.21.15.179:0/675254792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ec102770 msgr2=0x7f62ec109020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:07.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.129+0000 7f62f11a5700 1 --2- 172.21.15.179:0/675254792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ec102770 0x7f62ec109020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62e400b0d0 tx=0x7f62e400b3e0 comp rx=0 tx=0).stop 2024-02-14T17:45:07.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.129+0000 7f62f11a5700 1 -- 172.21.15.179:0/675254792 shutdown_connections 2024-02-14T17:45:07.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.129+0000 7f62f11a5700 1 --2- 172.21.15.179:0/675254792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ec102770 0x7f62ec109020 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:07.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.129+0000 7f62f11a5700 1 --2- 172.21.15.179:0/675254792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec100bd0 0x7f62ec100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:07.131 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.129+0000 7f62f11a5700 1 --2- 172.21.15.179:0/675254792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec101dd0 0x7f62ec102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:07.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.129+0000 7f62f11a5700 1 -- 172.21.15.179:0/675254792 >> 172.21.15.179:0/675254792 conn(0x7f62ec0fc170 msgr2=0x7f62ec0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:07.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.129+0000 7f62f11a5700 1 -- 172.21.15.179:0/675254792 shutdown_connections 2024-02-14T17:45:07.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.129+0000 7f62f11a5700 1 -- 172.21.15.179:0/675254792 wait complete. 2024-02-14T17:45:07.132 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62f11a5700 1 Processor -- start 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62f11a5700 1 -- start start 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62f11a5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ec100bd0 0x7f62ec197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62f11a5700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec101dd0 0x7f62ec197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62f11a5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec102770 0x7f62ec19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62f11a5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec19ed30 con 0x7f62ec102770 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62f11a5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec19ee70 con 0x7f62ec100bd0 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62f11a5700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec19f000 con 0x7f62ec101dd0 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62ead9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ec100bd0 0x7f62ec197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62ead9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ec100bd0 0x7f62ec197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56770/0 (socket says 172.21.15.179:56770) 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62ead9d700 1 -- 172.21.15.179:0/1825427589 learned_addr learned my addr 172.21.15.179:0/1825427589 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:45:07.134 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62ea59c700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec101dd0 0x7f62ec197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.130+0000 7f62eb59e700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec102770 0x7f62ec19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62ead9d700 1 -- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec101dd0 msgr2=0x7f62ec197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62ead9d700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec101dd0 0x7f62ec197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62ead9d700 1 -- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec102770 msgr2=0x7f62ec19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62ead9d700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec102770 0x7f62ec19e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62ead9d700 1 -- 172.21.15.179:0/1825427589 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e4009d00 con 0x7f62ec100bd0 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62eb59e700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec102770 0x7f62ec19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62ead9d700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ec100bd0 0x7f62ec197240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00d980 tx=0x7f62dc00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62d3fff700 1 -- 172.21.15.179:0/1825427589 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00de60 con 0x7f62ec100bd0 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62f11a5700 1 -- 172.21.15.179:0/1825427589 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec19f2e0 con 0x7f62ec100bd0 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62f11a5700 1 -- 172.21.15.179:0/1825427589 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec19f830 con 0x7f62ec100bd0 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62d3fff700 1 -- 172.21.15.179:0/1825427589 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f62dc0044a0 con 0x7f62ec100bd0 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.131+0000 7f62d3fff700 1 -- 172.21.15.179:0/1825427589 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc003a70 con 0x7f62ec100bd0 2024-02-14T17:45:07.135 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.132+0000 7f62f11a5700 1 -- 172.21.15.179:0/1825427589 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec1089b0 con 0x7f62ec100bd0 2024-02-14T17:45:07.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.133+0000 7f62d3fff700 1 -- 172.21.15.179:0/1825427589 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7f62dc003bd0 con 0x7f62ec100bd0 2024-02-14T17:45:07.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.133+0000 7f62d3fff700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f62d406d9d0 0x7f62d406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:07.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.133+0000 7f62d3fff700 1 -- 172.21.15.179:0/1825427589 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f62dc08d3b0 con 0x7f62ec100bd0 2024-02-14T17:45:07.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.133+0000 7f62ea59c700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f62d406d9d0 0x7f62d406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:07.136 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.134+0000 7f62ea59c700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f62d406d9d0 0x7f62d406fe90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62e0005ea0 tx=0x7f62e0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:07.143 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.141+0000 7f62d3fff700 1 -- 172.21.15.179:0/1825427589 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62dc056910 con 0x7f62ec100bd0 2024-02-14T17:45:07.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: osdmap e68: 8 total, 8 up, 8 in 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: Deploying daemon iscsi.iscsi.a on smithi179 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[39557]: pgmap v202: 132 pgs: 100 active+clean, 16 creating+peering, 16 unknown; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:45:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:45:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-14T17:45:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: osdmap e68: 8 total, 8 up, 8 in 2024-02-14T17:45:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-14T17:45:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-14T17:45:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: Deploying daemon iscsi.iscsi.a on smithi179 2024-02-14T17:45:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:06 smithi100 ceph-mon[30138]: pgmap v202: 132 pgs: 100 active+clean, 16 creating+peering, 16 unknown; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:45:07.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.362+0000 7f62f11a5700 1 -- 172.21.15.179:0/1825427589 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f62ec0620b0 con 0x7f62d406d9d0 2024-02-14T17:45:07.373 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.372+0000 7f62d3fff700 1 -- 172.21.15.179:0/1825427589 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f62ec0620b0 con 0x7f62d406d9d0 2024-02-14T17:45:07.374 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled prometheus update... 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 -- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f62d406d9d0 msgr2=0x7f62d406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f62d406d9d0 0x7f62d406fe90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62e0005ea0 tx=0x7f62e0005e30 comp rx=0 tx=0).stop 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 -- 172.21.15.179:0/1825427589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ec100bd0 msgr2=0x7f62ec197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ec100bd0 0x7f62ec197240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00d980 tx=0x7f62dc00dd40 comp rx=0 tx=0).stop 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 -- 172.21.15.179:0/1825427589 shutdown_connections 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f62d406d9d0 0x7f62d406fe90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62ec100bd0 0x7f62ec197240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62ec102770 0x7f62ec19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 --2- 172.21.15.179:0/1825427589 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f62ec101dd0 0x7f62ec197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 -- 172.21.15.179:0/1825427589 >> 172.21.15.179:0/1825427589 conn(0x7f62ec0fc170 msgr2=0x7f62ec0fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 -- 172.21.15.179:0/1825427589 shutdown_connections 2024-02-14T17:45:07.376 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:07.374+0000 7f62f11a5700 1 -- 172.21.15.179:0/1825427589 wait complete. 2024-02-14T17:45:07.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:07 smithi179 ceph-mon[33633]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-14T17:45:07.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:07 smithi179 ceph-mon[33633]: Cluster is now healthy 2024-02-14T17:45:07.946 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:07 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:08.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:07 smithi100 ceph-mon[30138]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-14T17:45:08.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:07 smithi100 ceph-mon[30138]: Cluster is now healthy 2024-02-14T17:45:08.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:07 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:08.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:07 smithi100 ceph-mon[39557]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-14T17:45:08.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:07 smithi100 ceph-mon[39557]: Cluster is now healthy 2024-02-14T17:45:08.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:07 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:08.515 DEBUG:teuthology.orchestra.run.smithi179:prometheus.a> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@prometheus.a.service 2024-02-14T17:45:08.518 INFO:tasks.cephadm:Adding node-exporter.a on smithi100 2024-02-14T17:45:08.518 INFO:tasks.cephadm:Adding node-exporter.b on smithi179 2024-02-14T17:45:08.518 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply node-exporter '2;smithi100=a;smithi179=b' 2024-02-14T17:45:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:08 smithi179 ceph-mon[33633]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:08 smithi179 ceph-mon[33633]: Saving service prometheus spec with placement smithi179=a;count:1 2024-02-14T17:45:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:08 smithi179 ceph-mon[33633]: pgmap v203: 132 pgs: 115 active+clean, 12 creating+peering, 5 unknown; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 3.3 KiB/s wr, 74 op/s 2024-02-14T17:45:09.026 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:45:09.051 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:08 smithi100 ceph-mon[30138]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:09.052 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:08 smithi100 ceph-mon[30138]: Saving service prometheus spec with placement smithi179=a;count:1 2024-02-14T17:45:09.052 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:08 smithi100 ceph-mon[30138]: pgmap v203: 132 pgs: 115 active+clean, 12 creating+peering, 5 unknown; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 3.3 KiB/s wr, 74 op/s 2024-02-14T17:45:09.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:08 smithi100 ceph-mon[39557]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:09.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:08 smithi100 ceph-mon[39557]: Saving service prometheus spec with placement smithi179=a;count:1 2024-02-14T17:45:09.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:08 smithi100 ceph-mon[39557]: pgmap v203: 132 pgs: 115 active+clean, 12 creating+peering, 5 unknown; 580 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 31 KiB/s rd, 3.3 KiB/s wr, 74 op/s 2024-02-14T17:45:09.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:09 smithi179 systemd[1]: Starting Ceph iscsi.iscsi.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:45:10.466 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:10 smithi179 ceph-mon[33633]: osdmap e69: 8 total, 8 up, 8 in 2024-02-14T17:45:10.466 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:10 smithi179 ceph-mon[33633]: pgmap v205: 132 pgs: 132 active+clean; 581 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.7 KiB/s wr, 132 op/s 2024-02-14T17:45:10.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:10 smithi100 ceph-mon[30138]: osdmap e69: 8 total, 8 up, 8 in 2024-02-14T17:45:10.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:10 smithi100 ceph-mon[30138]: pgmap v205: 132 pgs: 132 active+clean; 581 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.7 KiB/s wr, 132 op/s 2024-02-14T17:45:10.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:10 smithi100 ceph-mon[39557]: osdmap e69: 8 total, 8 up, 8 in 2024-02-14T17:45:10.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:10 smithi100 ceph-mon[39557]: pgmap v205: 132 pgs: 132 active+clean; 581 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.7 KiB/s wr, 132 op/s 2024-02-14T17:45:10.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.687+0000 7f197f828700 1 -- 172.21.15.179:0/2347529565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1978101da0 msgr2=0x7f1978102200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.687+0000 7f197f828700 1 --2- 172.21.15.179:0/2347529565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1978101da0 0x7f1978102200 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f19680099c0 tx=0x7f1968009cd0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.687+0000 7f197f828700 1 -- 172.21.15.179:0/2347529565 shutdown_connections 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.687+0000 7f197f828700 1 --2- 172.21.15.179:0/2347529565 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1978102740 0x7f1978108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.687+0000 7f197f828700 1 --2- 172.21.15.179:0/2347529565 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1978100ba0 0x7f1978100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.687+0000 7f197f828700 1 --2- 172.21.15.179:0/2347529565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1978101da0 0x7f1978102200 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.687+0000 7f197f828700 1 -- 172.21.15.179:0/2347529565 >> 172.21.15.179:0/2347529565 conn(0x7f19780fc140 msgr2=0x7f19780fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.687+0000 7f197f828700 1 -- 172.21.15.179:0/2347529565 shutdown_connections 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.687+0000 7f197f828700 1 -- 172.21.15.179:0/2347529565 wait complete. 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197f828700 1 Processor -- start 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197f828700 1 -- start start 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197f828700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1978100ba0 0x7f197810b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197f828700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1978101da0 0x7f1978109b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:10.691 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197f828700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1978102740 0x7f197810a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197f828700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197810a7c0 con 0x7f1978101da0 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197f828700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197810a930 con 0x7f1978100ba0 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197f828700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19781a1220 con 0x7f1978102740 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197d5c4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1978100ba0 0x7f197810b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197d5c4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1978100ba0 0x7f197810b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:56800/0 (socket says 172.21.15.179:56800) 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197d5c4700 1 -- 172.21.15.179:0/2126247974 learned_addr learned my addr 172.21.15.179:0/2126247974 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197cdc3700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1978101da0 0x7f1978109b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197ddc5700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1978102740 0x7f197810a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197d5c4700 1 -- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1978102740 msgr2=0x7f197810a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197d5c4700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1978102740 0x7f197810a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197d5c4700 1 -- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1978101da0 msgr2=0x7f1978109b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197d5c4700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1978101da0 0x7f1978109b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.692 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197d5c4700 1 -- 172.21.15.179:0/2126247974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680095f0 con 0x7f1978100ba0 2024-02-14T17:45:10.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f197d5c4700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1978100ba0 0x7f197810b4b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1974007ee0 tx=0x7f197400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:10.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.688+0000 7f196e7fc700 1 -- 172.21.15.179:0/2126247974 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197400ddd0 con 0x7f1978100ba0 2024-02-14T17:45:10.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.689+0000 7f196e7fc700 1 -- 172.21.15.179:0/2126247974 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f197400f040 con 0x7f1978100ba0 2024-02-14T17:45:10.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.689+0000 7f197f828700 1 -- 172.21.15.179:0/2126247974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19781a1500 con 0x7f1978100ba0 2024-02-14T17:45:10.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.689+0000 7f196e7fc700 1 -- 172.21.15.179:0/2126247974 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974014600 con 0x7f1978100ba0 2024-02-14T17:45:10.693 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.689+0000 7f197f828700 1 -- 172.21.15.179:0/2126247974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19781a1a50 con 0x7f1978100ba0 2024-02-14T17:45:10.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.690+0000 7f197f828700 1 -- 172.21.15.179:0/2126247974 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1978108050 con 0x7f1978100ba0 2024-02-14T17:45:10.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.690+0000 7f196e7fc700 1 -- 172.21.15.179:0/2126247974 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) v1 ==== 91407+0+0 (secure 0 0 0) 0x7f1974015400 con 0x7f1978100ba0 2024-02-14T17:45:10.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.690+0000 7f196e7fc700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f196406d9d0 0x7f196406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:10.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.690+0000 7f196e7fc700 1 -- 172.21.15.179:0/2126247974 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f197408ce80 con 0x7f1978100ba0 2024-02-14T17:45:10.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.690+0000 7f197cdc3700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f196406d9d0 0x7f196406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:10.694 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.691+0000 7f197cdc3700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f196406d9d0 0x7f196406fe90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1968009990 tx=0x7f1968005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:10.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.696+0000 7f196e7fc700 1 -- 172.21.15.179:0/2126247974 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1974056480 con 0x7f1978100ba0 2024-02-14T17:45:10.910 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.909+0000 7f197f828700 1 -- 172.21.15.179:0/2126247974 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi179=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f197810cc60 con 0x7f196406d9d0 2024-02-14T17:45:10.917 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.916+0000 7f196e7fc700 1 -- 172.21.15.179:0/2126247974 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f197810cc60 con 0x7f196406d9d0 2024-02-14T17:45:10.918 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled node-exporter update... 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 -- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f196406d9d0 msgr2=0x7f196406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f196406d9d0 0x7f196406fe90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1968009990 tx=0x7f1968005e80 comp rx=0 tx=0).stop 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 -- 172.21.15.179:0/2126247974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1978100ba0 msgr2=0x7f197810b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1978100ba0 0x7f197810b4b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1974007ee0 tx=0x7f197400d3b0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 -- 172.21.15.179:0/2126247974 shutdown_connections 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f196406d9d0 0x7f196406fe90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1978100ba0 0x7f197810b4b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1978102740 0x7f197810a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 --2- 172.21.15.179:0/2126247974 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1978101da0 0x7f1978109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 -- 172.21.15.179:0/2126247974 >> 172.21.15.179:0/2126247974 conn(0x7f19780fc140 msgr2=0x7f1978105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:10.920 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 -- 172.21.15.179:0/2126247974 shutdown_connections 2024-02-14T17:45:10.921 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:10.918+0000 7f197f828700 1 -- 172.21.15.179:0/2126247974 wait complete. 2024-02-14T17:45:11.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:10 smithi179 podman[62561]: 2024-02-14 17:45:10.807700861 +0000 UTC m=+0.104134576 container create a7ccf93cf909e2dcf07c072be15c555c9a5ac3e6bb3b08139ab723be3e7aadb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T17:45:11.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:10 smithi179 podman[62561]: 2024-02-14 17:45:10.721420226 +0000 UTC m=+0.017853955 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T17:45:11.507 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:11 smithi179 podman[62561]: 2024-02-14 17:45:11.244912848 +0000 UTC m=+0.541346563 container init a7ccf93cf909e2dcf07c072be15c555c9a5ac3e6bb3b08139ab723be3e7aadb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-14T17:45:11.507 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:11 smithi179 podman[62561]: 2024-02-14 17:45:11.250481239 +0000 UTC m=+0.546914963 container start a7ccf93cf909e2dcf07c072be15c555c9a5ac3e6bb3b08139ab723be3e7aadb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-14T17:45:11.507 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:11 smithi179 bash[62561]: a7ccf93cf909e2dcf07c072be15c555c9a5ac3e6bb3b08139ab723be3e7aadb9 2024-02-14T17:45:11.508 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:11 smithi179 systemd[1]: Started Ceph iscsi.iscsi.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:45:11.792 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug Started the configuration object watcher 2024-02-14T17:45:11.792 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug Checking for config object changes every 1s 2024-02-14T17:45:11.792 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug Processing osd blocklist entries for this node 2024-02-14T17:45:11.820 DEBUG:teuthology.orchestra.run.smithi100:node-exporter.a> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@node-exporter.a.service 2024-02-14T17:45:11.822 DEBUG:teuthology.orchestra.run.smithi179:node-exporter.b> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@node-exporter.b.service 2024-02-14T17:45:11.824 INFO:tasks.cephadm:Adding alertmanager.a on smithi100 2024-02-14T17:45:11.825 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply alertmanager '1;smithi100=a' 2024-02-14T17:45:12.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug Reading the configuration object to update local LIO configuration 2024-02-14T17:45:12.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug Configuration does not have an entry for this host(smithi179.front.sepia.ceph.com) - nothing to define to LIO 2024-02-14T17:45:12.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-14T17:45:12.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-mon[33633]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:12.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-mon[33633]: Saving service node-exporter spec with placement smithi100=a;smithi179=b;count:2 2024-02-14T17:45:12.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:12.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/402430125' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T17:45:12.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[30138]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:12.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[30138]: Saving service node-exporter spec with placement smithi100=a;smithi179=b;count:2 2024-02-14T17:45:12.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/402430125' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T17:45:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[39557]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi100=a;smithi179=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[39557]: Saving service node-exporter spec with placement smithi100=a;smithi179=b;count:2 2024-02-14T17:45:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:12.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:12.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/402430125' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-14T17:45:12.439 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:45:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:12 smithi179 ceph-mon[33633]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-14T17:45:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:12 smithi179 ceph-mon[33633]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 4.9 KiB/s wr, 152 op/s 2024-02-14T17:45:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:12 smithi179 ceph-mon[33633]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-14T17:45:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:12 smithi100 ceph-mon[30138]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-14T17:45:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:12 smithi100 ceph-mon[30138]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 4.9 KiB/s wr, 152 op/s 2024-02-14T17:45:13.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:12 smithi100 ceph-mon[30138]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-14T17:45:13.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:12 smithi100 ceph-mon[39557]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-14T17:45:13.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:12 smithi100 ceph-mon[39557]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s rd, 4.9 KiB/s wr, 152 op/s 2024-02-14T17:45:13.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:12 smithi100 ceph-mon[39557]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-14T17:45:14.276 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:14 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:14.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:14 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:14 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:14.613 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.274+0000 7fce3fdb1700 1 -- 172.21.15.179:0/1239986885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce38101de0 msgr2=0x7fce38102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.274+0000 7fce3fdb1700 1 --2- 172.21.15.179:0/1239986885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce38101de0 0x7fce38102260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fce34009990 tx=0x7fce34009ca0 comp rx=0 tx=0).stop 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.274+0000 7fce3fdb1700 1 -- 172.21.15.179:0/1239986885 shutdown_connections 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.274+0000 7fce3fdb1700 1 --2- 172.21.15.179:0/1239986885 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce381027a0 0x7fce38109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.274+0000 7fce3fdb1700 1 --2- 172.21.15.179:0/1239986885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce381010c0 0x7fce381014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.274+0000 7fce3fdb1700 1 --2- 172.21.15.179:0/1239986885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce38101de0 0x7fce38102260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.274+0000 7fce3fdb1700 1 -- 172.21.15.179:0/1239986885 >> 172.21.15.179:0/1239986885 conn(0x7fce380fb8e0 msgr2=0x7fce380fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.274+0000 7fce3fdb1700 1 -- 172.21.15.179:0/1239986885 shutdown_connections 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.274+0000 7fce3fdb1700 1 -- 172.21.15.179:0/1239986885 wait complete. 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3fdb1700 1 Processor -- start 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3fdb1700 1 -- start start 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3fdb1700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce381010c0 0x7fce381007a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:14.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3fdb1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce38101de0 0x7fce3819c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3fdb1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce381027a0 0x7fce3819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3fdb1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3819ee60 con 0x7fce381027a0 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3fdb1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3819efd0 con 0x7fce38101de0 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3fdb1700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3819f190 con 0x7fce381010c0 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3d34c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce38101de0 0x7fce3819c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3d34c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce38101de0 0x7fce3819c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40632/0 (socket says 172.21.15.179:40632) 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.275+0000 7fce3d34c700 1 -- 172.21.15.179:0/487795185 learned_addr learned my addr 172.21.15.179:0/487795185 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce3db4d700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce381010c0 0x7fce381007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce3e34e700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce381027a0 0x7fce3819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce3d34c700 1 -- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce381010c0 msgr2=0x7fce381007a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce3d34c700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce381010c0 0x7fce381007a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce3d34c700 1 -- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce381027a0 msgr2=0x7fce3819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce3d34c700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce381027a0 0x7fce3819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:14.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce3d34c700 1 -- 172.21.15.179:0/487795185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce340095f0 con 0x7fce38101de0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce3d34c700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce38101de0 0x7fce3819c070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce34004800 tx=0x7fce3400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce2effd700 1 -- 172.21.15.179:0/487795185 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3401d070 con 0x7fce38101de0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce2effd700 1 -- 172.21.15.179:0/487795185 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fce3400bd10 con 0x7fce38101de0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce3fdb1700 1 -- 172.21.15.179:0/487795185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3819f410 con 0x7fce38101de0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce2effd700 1 -- 172.21.15.179:0/487795185 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34022db0 con 0x7fce38101de0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.276+0000 7fce3fdb1700 1 -- 172.21.15.179:0/487795185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3819f900 con 0x7fce38101de0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.277+0000 7fce3fdb1700 1 -- 172.21.15.179:0/487795185 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce381077c0 con 0x7fce38101de0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.278+0000 7fce2effd700 1 -- 172.21.15.179:0/487795185 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 91458+0+0 (secure 0 0 0) 0x7fce34022600 con 0x7fce38101de0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.278+0000 7fce2effd700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fce2406da60 0x7fce2406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.278+0000 7fce2effd700 1 -- 172.21.15.179:0/487795185 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fce3408ec80 con 0x7fce38101de0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.278+0000 7fce3db4d700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fce2406da60 0x7fce2406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:14.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.279+0000 7fce3db4d700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fce2406da60 0x7fce2406ff20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fce38069490 tx=0x7fce28009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled alertmanager update... 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.283+0000 7fce2effd700 1 -- 172.21.15.179:0/487795185 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce340582a0 con 0x7fce38101de0 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.496+0000 7fce3fdb1700 1 -- 172.21.15.179:0/487795185 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi100=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fce3810bd10 con 0x7fce2406da60 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.504+0000 7fce2effd700 1 -- 172.21.15.179:0/487795185 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fce3810bd10 con 0x7fce2406da60 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 -- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fce2406da60 msgr2=0x7fce2406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fce2406da60 0x7fce2406ff20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fce38069490 tx=0x7fce28009190 comp rx=0 tx=0).stop 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 -- 172.21.15.179:0/487795185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce38101de0 msgr2=0x7fce3819c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce38101de0 0x7fce3819c070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce34004800 tx=0x7fce3400f690 comp rx=0 tx=0).stop 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 -- 172.21.15.179:0/487795185 shutdown_connections 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fce381027a0 0x7fce3819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:14.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fce381010c0 0x7fce381007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:14.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fce2406da60 0x7fce2406ff20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:14.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 --2- 172.21.15.179:0/487795185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fce38101de0 0x7fce3819c070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:14.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 -- 172.21.15.179:0/487795185 >> 172.21.15.179:0/487795185 conn(0x7fce380fb8e0 msgr2=0x7fce380fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:14.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 -- 172.21.15.179:0/487795185 shutdown_connections 2024-02-14T17:45:14.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:14.506+0000 7fce3fdb1700 1 -- 172.21.15.179:0/487795185 wait complete. 2024-02-14T17:45:15.266 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:15 smithi179 ceph-mon[33633]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.2 KiB/s wr, 133 op/s 2024-02-14T17:45:15.266 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:15 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:15.277 DEBUG:teuthology.orchestra.run.smithi100:alertmanager.a> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@alertmanager.a.service 2024-02-14T17:45:15.280 INFO:tasks.cephadm:Adding grafana.a on smithi179 2024-02-14T17:45:15.280 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph orch apply grafana '1;smithi179=a' 2024-02-14T17:45:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:15 smithi100 ceph-mon[30138]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.2 KiB/s wr, 133 op/s 2024-02-14T17:45:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:15 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:15.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:15 smithi100 ceph-mon[39557]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.2 KiB/s wr, 133 op/s 2024-02-14T17:45:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:15 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:15.820 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:45:16.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:16 smithi179 ceph-mon[33633]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi100=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:16.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:16 smithi179 ceph-mon[33633]: Saving service alertmanager spec with placement smithi100=a;count:1 2024-02-14T17:45:16.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:16 smithi100 ceph-mon[30138]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi100=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:16.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:16 smithi100 ceph-mon[30138]: Saving service alertmanager spec with placement smithi100=a;count:1 2024-02-14T17:45:16.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:16 smithi100 ceph-mon[39557]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi100=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:16.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:16 smithi100 ceph-mon[39557]: Saving service alertmanager spec with placement smithi100=a;count:1 2024-02-14T17:45:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:17 smithi179 ceph-mon[33633]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-02-14T17:45:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:45:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:17 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:17.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[30138]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-02-14T17:45:17.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:17.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:17.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:17.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:45:17.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:17.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[39557]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 113 op/s 2024-02-14T17:45:17.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:17.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:17.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:45:17.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:45:17.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:17 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.611+0000 7fed6db19700 1 -- 172.21.15.179:0/53979743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed68100e90 msgr2=0x7fed681012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.611+0000 7fed6db19700 1 --2- 172.21.15.179:0/53979743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed68100e90 0x7fed681012f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009990 tx=0x7fed5c009ca0 comp rx=0 tx=0).stop 2024-02-14T17:45:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.611+0000 7fed6db19700 1 -- 172.21.15.179:0/53979743 shutdown_connections 2024-02-14T17:45:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.611+0000 7fed6db19700 1 --2- 172.21.15.179:0/53979743 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed68101830 0x7fed681090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.611+0000 7fed6db19700 1 --2- 172.21.15.179:0/53979743 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed68102150 0x7fed68102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.611+0000 7fed6db19700 1 --2- 172.21.15.179:0/53979743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed68100e90 0x7fed681012f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:17.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.611+0000 7fed6db19700 1 -- 172.21.15.179:0/53979743 >> 172.21.15.179:0/53979743 conn(0x7fed680fb900 msgr2=0x7fed680fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.611+0000 7fed6db19700 1 -- 172.21.15.179:0/53979743 shutdown_connections 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.611+0000 7fed6db19700 1 -- 172.21.15.179:0/53979743 wait complete. 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.612+0000 7fed6db19700 1 Processor -- start 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.612+0000 7fed6db19700 1 -- start start 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.612+0000 7fed6db19700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed68100e90 0x7fed68100890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.612+0000 7fed6db19700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed68101830 0x7fed6819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.612+0000 7fed6db19700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed68102150 0x7fed681a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.612+0000 7fed6db19700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed681a0fa0 con 0x7fed68101830 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.612+0000 7fed6db19700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed681a1110 con 0x7fed68102150 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.612+0000 7fed6db19700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed681a12d0 con 0x7fed68100e90 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.612+0000 7fed67fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed68102150 0x7fed681a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed677fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed68100e90 0x7fed68100890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:17.616 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed67fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed68102150 0x7fed681a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:40652/0 (socket says 172.21.15.179:40652) 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed67fff700 1 -- 172.21.15.179:0/2745881908 learned_addr learned my addr 172.21.15.179:0/2745881908 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed66ffd700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed68101830 0x7fed6819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed67fff700 1 -- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed68100e90 msgr2=0x7fed68100890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed67fff700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed68100e90 0x7fed68100890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed67fff700 1 -- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed68101830 msgr2=0x7fed6819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed67fff700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed68101830 0x7fed6819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed67fff700 1 -- 172.21.15.179:0/2745881908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c0095f0 con 0x7fed68102150 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed67fff700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed68102150 0x7fed681a0980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed6000b970 tx=0x7fed6000bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed64ff9700 1 -- 172.21.15.179:0/2745881908 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6000d860 con 0x7fed68102150 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed64ff9700 1 -- 172.21.15.179:0/2745881908 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fed6000de40 con 0x7fed68102150 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed6db19700 1 -- 172.21.15.179:0/2745881908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed681a15b0 con 0x7fed68102150 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed64ff9700 1 -- 172.21.15.179:0/2745881908 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6000ee70 con 0x7fed68102150 2024-02-14T17:45:17.617 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.613+0000 7fed6db19700 1 -- 172.21.15.179:0/2745881908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed681a1b30 con 0x7fed68102150 2024-02-14T17:45:17.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.614+0000 7fed6db19700 1 -- 172.21.15.179:0/2745881908 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed68107880 con 0x7fed68102150 2024-02-14T17:45:17.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.615+0000 7fed64ff9700 1 -- 172.21.15.179:0/2745881908 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 91458+0+0 (secure 0 0 0) 0x7fed6000d9c0 con 0x7fed68102150 2024-02-14T17:45:17.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.615+0000 7fed64ff9700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed5006da60 0x7fed5006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:17.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.615+0000 7fed64ff9700 1 -- 172.21.15.179:0/2745881908 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fed60006e40 con 0x7fed68102150 2024-02-14T17:45:17.618 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.615+0000 7fed677fe700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed5006da60 0x7fed5006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:17.619 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.616+0000 7fed677fe700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed5006da60 0x7fed5006ff20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fed6806a150 tx=0x7fed58009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:17.621 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.620+0000 7fed64ff9700 1 -- 172.21.15.179:0/2745881908 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed600567d0 con 0x7fed68102150 2024-02-14T17:45:17.826 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.824+0000 7fed6db19700 1 -- 172.21.15.179:0/2745881908 --> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fed6810bd00 con 0x7fed5006da60 2024-02-14T17:45:17.833 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.831+0000 7fed64ff9700 1 -- 172.21.15.179:0/2745881908 <== mgr.14152 v2:172.21.15.100:6800/3957561430 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fed6810bd00 con 0x7fed5006da60 2024-02-14T17:45:17.833 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled grafana update... 2024-02-14T17:45:17.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 -- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed5006da60 msgr2=0x7fed5006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:17.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed5006da60 0x7fed5006ff20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fed6806a150 tx=0x7fed58009190 comp rx=0 tx=0).stop 2024-02-14T17:45:17.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 -- 172.21.15.179:0/2745881908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed68102150 msgr2=0x7fed681a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:17.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed68102150 0x7fed681a0980 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed6000b970 tx=0x7fed6000bc80 comp rx=0 tx=0).stop 2024-02-14T17:45:17.836 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 -- 172.21.15.179:0/2745881908 shutdown_connections 2024-02-14T17:45:17.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fed5006da60 0x7fed5006ff20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:17.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed68102150 0x7fed681a0980 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:17.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fed68100e90 0x7fed68100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:17.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 --2- 172.21.15.179:0/2745881908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fed68101830 0x7fed6819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:17.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 -- 172.21.15.179:0/2745881908 >> 172.21.15.179:0/2745881908 conn(0x7fed680fb900 msgr2=0x7fed680fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:17.837 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 -- 172.21.15.179:0/2745881908 shutdown_connections 2024-02-14T17:45:17.838 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:17.834+0000 7fed6db19700 1 -- 172.21.15.179:0/2745881908 wait complete. 2024-02-14T17:45:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[30138]: Deploying daemon prometheus.a on smithi179 2024-02-14T17:45:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-14T17:45:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-02-14T17:45:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-14T17:45:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:18.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[39557]: Deploying daemon prometheus.a on smithi179 2024-02-14T17:45:18.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:18.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-14T17:45:18.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-02-14T17:45:18.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-14T17:45:18.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:18 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:18.480 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:18 smithi179 ceph-mon[33633]: Deploying daemon prometheus.a on smithi179 2024-02-14T17:45:18.480 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:18 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:18.481 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:18 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-14T17:45:18.481 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:18 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-02-14T17:45:18.481 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:18 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-02-14T17:45:18.481 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:18 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:18.909 DEBUG:teuthology.orchestra.run.smithi179:grafana.a> sudo journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@grafana.a.service 2024-02-14T17:45:18.911 INFO:tasks.cephadm:Setting up client nodes... 2024-02-14T17:45:18.912 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-14T17:45:19.160 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:19.282 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[30138]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:19.283 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[30138]: Saving service grafana spec with placement smithi179=a;count:1 2024-02-14T17:45:19.283 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[30138]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 2.1 KiB/s wr, 74 op/s 2024-02-14T17:45:19.283 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-14T17:45:19.283 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-02-14T17:45:19.283 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-02-14T17:45:19.283 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[30138]: osdmap e70: 8 total, 8 up, 8 in 2024-02-14T17:45:19.283 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[39557]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:19.283 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[39557]: Saving service grafana spec with placement smithi179=a;count:1 2024-02-14T17:45:19.283 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[39557]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 2.1 KiB/s wr, 74 op/s 2024-02-14T17:45:19.284 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-14T17:45:19.284 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-02-14T17:45:19.284 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-02-14T17:45:19.284 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:19 smithi100 ceph-mon[39557]: osdmap e70: 8 total, 8 up, 8 in 2024-02-14T17:45:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:19 smithi179 ceph-mon[33633]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi179=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-14T17:45:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:19 smithi179 ceph-mon[33633]: Saving service grafana spec with placement smithi179=a;count:1 2024-02-14T17:45:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:19 smithi179 ceph-mon[33633]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 2.1 KiB/s wr, 74 op/s 2024-02-14T17:45:19.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:19 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-14T17:45:19.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:19 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-02-14T17:45:19.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:19 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-02-14T17:45:19.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:19 smithi179 ceph-mon[33633]: osdmap e70: 8 total, 8 up, 8 in 2024-02-14T17:45:20.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.968+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/1093605553 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdba8101dc0 msgr2=0x7fdba8102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:20.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.968+0000 7fdbaf8fc700 1 --2- 172.21.15.100:0/1093605553 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdba8101dc0 0x7fdba8102220 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009990 tx=0x7fdb98009ca0 comp rx=0 tx=0).stop 2024-02-14T17:45:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.968+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/1093605553 shutdown_connections 2024-02-14T17:45:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.968+0000 7fdbaf8fc700 1 --2- 172.21.15.100:0/1093605553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdba8100bc0 0x7fdba8100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.968+0000 7fdbaf8fc700 1 --2- 172.21.15.100:0/1093605553 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdba8102760 0x7fdba8109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.968+0000 7fdbaf8fc700 1 --2- 172.21.15.100:0/1093605553 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdba8101dc0 0x7fdba8102220 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.968+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/1093605553 >> 172.21.15.100:0/1093605553 conn(0x7fdba80fc160 msgr2=0x7fdba80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.968+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/1093605553 shutdown_connections 2024-02-14T17:45:20.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.968+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/1093605553 wait complete. 2024-02-14T17:45:20.971 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:20 smithi100 ceph-mon[30138]: osdmap e71: 8 total, 8 up, 8 in 2024-02-14T17:45:20.971 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:20 smithi100 ceph-mon[30138]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 127 B/s wr, 1 op/s 2024-02-14T17:45:20.972 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:20 smithi100 ceph-mon[39557]: osdmap e71: 8 total, 8 up, 8 in 2024-02-14T17:45:20.972 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:20 smithi100 ceph-mon[39557]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 127 B/s wr, 1 op/s 2024-02-14T17:45:20.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbaf8fc700 1 Processor -- start 2024-02-14T17:45:20.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbaf8fc700 1 -- start start 2024-02-14T17:45:20.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbaf8fc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdba8100bc0 0x7fdba8196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:20.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbaf8fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdba8101dc0 0x7fdba8196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:20.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbaf8fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdba8102760 0x7fdba819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:20.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbaf8fc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba819db40 con 0x7fdba8102760 2024-02-14T17:45:20.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbaf8fc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba819dc80 con 0x7fdba8101dc0 2024-02-14T17:45:20.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbaf8fc700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba819de10 con 0x7fdba8100bc0 2024-02-14T17:45:20.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbad698700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdba8100bc0 0x7fdba8196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbad698700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdba8100bc0 0x7fdba8196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:47000/0 (socket says 172.21.15.100:47000) 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbad698700 1 -- 172.21.15.100:0/3070854378 learned_addr learned my addr 172.21.15.100:0/3070854378 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbade99700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdba8102760 0x7fdba819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbace97700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdba8101dc0 0x7fdba8196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbad698700 1 -- 172.21.15.100:0/3070854378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdba8101dc0 msgr2=0x7fdba8196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbad698700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdba8101dc0 0x7fdba8196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbad698700 1 -- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdba8102760 msgr2=0x7fdba819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbad698700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdba8102760 0x7fdba819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbad698700 1 -- 172.21.15.100:0/3070854378 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba0009520 con 0x7fdba8100bc0 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbade99700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdba8102760 0x7fdba819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.969+0000 7fdbad698700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdba8100bc0 0x7fdba8196000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdba4007f40 tx=0x7fdba400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.970+0000 7fdb9e7fc700 1 -- 172.21.15.100:0/3070854378 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4017070 con 0x7fdba8100bc0 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.970+0000 7fdb9e7fc700 1 -- 172.21.15.100:0/3070854378 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdba400f040 con 0x7fdba8100bc0 2024-02-14T17:45:20.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.970+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/3070854378 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb980095f0 con 0x7fdba8100bc0 2024-02-14T17:45:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.970+0000 7fdb9e7fc700 1 -- 172.21.15.100:0/3070854378 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba40043c0 con 0x7fdba8100bc0 2024-02-14T17:45:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.970+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/3070854378 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba819e4b0 con 0x7fdba8100bc0 2024-02-14T17:45:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.971+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/3070854378 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba81077a0 con 0x7fdba8100bc0 2024-02-14T17:45:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.971+0000 7fdb9e7fc700 1 -- 172.21.15.100:0/3070854378 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 16) v1 ==== 91458+0+0 (secure 0 0 0) 0x7fdba400dbd0 con 0x7fdba8100bc0 2024-02-14T17:45:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.972+0000 7fdb9e7fc700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fdb9406db10 0x7fdb9406ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.972+0000 7fdb9e7fc700 1 -- 172.21.15.100:0/3070854378 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7207+0+0 (secure 0 0 0) 0x7fdba408cdd0 con 0x7fdba8100bc0 2024-02-14T17:45:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.972+0000 7fdbace97700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fdb9406db10 0x7fdb9406ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:20.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.972+0000 7fdbace97700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fdb9406db10 0x7fdb9406ffd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800b2e0 tx=0x7fdb9800b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:20.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:20.977+0000 7fdb9e7fc700 1 -- 172.21.15.100:0/3070854378 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdba4056290 con 0x7fdba8100bc0 2024-02-14T17:45:21.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.244+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/3070854378 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fdba8050180 con 0x7fdba8100bc0 2024-02-14T17:45:21.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.249+0000 7fdb9e7fc700 1 -- 172.21.15.100:0/3070854378 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fdba4059950 con 0x7fdba8100bc0 2024-02-14T17:45:21.251 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-02-14T17:45:21.251 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQAx/MxldHqzDhAA+0uD1arWeUf0MCBqB/yXTA== 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fdb9406db10 msgr2=0x7fdb9406ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fdb9406db10 0x7fdb9406ffd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800b2e0 tx=0x7fdb9800b610 comp rx=0 tx=0).stop 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdba8100bc0 msgr2=0x7fdba8196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdba8100bc0 0x7fdba8196000 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdba4007f40 tx=0x7fdba400d3b0 comp rx=0 tx=0).stop 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/3070854378 shutdown_connections 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdba8102760 0x7fdba819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fdba8100bc0 0x7fdba8196000 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7fdb9406db10 0x7fdb9406ffd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 --2- 172.21.15.100:0/3070854378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdba8101dc0 0x7fdba8196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/3070854378 >> 172.21.15.100:0/3070854378 conn(0x7fdba80fc160 msgr2=0x7fdba80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/3070854378 shutdown_connections 2024-02-14T17:45:21.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:21.251+0000 7fdbaf8fc700 1 -- 172.21.15.100:0/3070854378 wait complete. 2024-02-14T17:45:21.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:20 smithi179 ceph-mon[33633]: osdmap e71: 8 total, 8 up, 8 in 2024-02-14T17:45:21.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:20 smithi179 ceph-mon[33633]: pgmap v212: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 127 B/s wr, 1 op/s 2024-02-14T17:45:21.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:45:21.936 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:45:21.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-14T17:45:21.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-14T17:45:21.979 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-14T17:45:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3070854378' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:45:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3070854378' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:45:22.214 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.b/config 2024-02-14T17:45:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3070854378' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:45:23.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:22 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:23.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:22 smithi100 ceph-mon[30138]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 127 B/s wr, 1 op/s; 13 B/s, 1 objects/s recovering 2024-02-14T17:45:23.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:22 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:23.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:22 smithi100 ceph-mon[39557]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 127 B/s wr, 1 op/s; 13 B/s, 1 objects/s recovering 2024-02-14T17:45:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:22 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:22 smithi179 ceph-mon[33633]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 127 B/s wr, 1 op/s; 13 B/s, 1 objects/s recovering 2024-02-14T17:45:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:25 smithi179 ceph-mon[33633]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68 B/s, 1 objects/s recovering 2024-02-14T17:45:25.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:25 smithi100 ceph-mon[30138]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68 B/s, 1 objects/s recovering 2024-02-14T17:45:25.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:25 smithi100 ceph-mon[39557]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 68 B/s, 1 objects/s recovering 2024-02-14T17:45:27.221 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.691+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/455335467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b48101ce0 msgr2=0x7f6b48102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.691+0000 7f6b4d3f1700 1 --2- 172.21.15.179:0/455335467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b48101ce0 0x7f6b48102100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b380099c0 tx=0x7f6b38009cd0 comp rx=0 tx=0).stop 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.691+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/455335467 shutdown_connections 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.691+0000 7f6b4d3f1700 1 --2- 172.21.15.179:0/455335467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b48101ce0 0x7f6b48102100 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.691+0000 7f6b4d3f1700 1 --2- 172.21.15.179:0/455335467 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b48103880 0x7f6b4810a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.691+0000 7f6b4d3f1700 1 --2- 172.21.15.179:0/455335467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b48102ee0 0x7f6b48103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.691+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/455335467 >> 172.21.15.179:0/455335467 conn(0x7f6b480fd2c0 msgr2=0x7f6b480ff6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.691+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/455335467 shutdown_connections 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.691+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/455335467 wait complete. 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.691+0000 7f6b4d3f1700 1 Processor -- start 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b4d3f1700 1 -- start start 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b4d3f1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b48101ce0 0x7f6b481971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:27.222 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b4d3f1700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b48102ee0 0x7f6b48197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b4d3f1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b48103880 0x7f6b4819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b4d3f1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4819ed50 con 0x7f6b48101ce0 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b4d3f1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4819ee90 con 0x7f6b48103880 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b4d3f1700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4819f020 con 0x7f6b48102ee0 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b477fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b48103880 0x7f6b4819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b477fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b48103880 0x7f6b4819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55624/0 (socket says 172.21.15.179:55624) 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b477fe700 1 -- 172.21.15.179:0/3111517023 learned_addr learned my addr 172.21.15.179:0/3111517023 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b46ffd700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b48101ce0 0x7f6b481971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b467fc700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b48102ee0 0x7f6b48197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b477fe700 1 -- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b48102ee0 msgr2=0x7f6b48197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b477fe700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b48102ee0 0x7f6b48197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b477fe700 1 -- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b48101ce0 msgr2=0x7f6b481971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b477fe700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b48101ce0 0x7f6b481971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:27.223 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b477fe700 1 -- 172.21.15.179:0/3111517023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b380095f0 con 0x7f6b48103880 2024-02-14T17:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b46ffd700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b48101ce0 0x7f6b481971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T17:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b467fc700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b48102ee0 0x7f6b48197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T17:45:27.224 INFO:teuthology.orchestra.run.smithi179.stdout:[client.1] 2024-02-14T17:45:27.224 INFO:teuthology.orchestra.run.smithi179.stdout: key = AQA2/Mxld55COhAAK61K8Bme6SuJxh84b1mU9Q== 2024-02-14T17:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b477fe700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b48103880 0x7f6b4819e780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000b970 tx=0x7f6b4000bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:27.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.692+0000 7f6b2ffff700 1 -- 172.21.15.179:0/3111517023 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4000c9b0 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.693+0000 7f6b2ffff700 1 -- 172.21.15.179:0/3111517023 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6b40014440 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.693+0000 7f6b2ffff700 1 -- 172.21.15.179:0/3111517023 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b400075d0 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.693+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/3111517023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4819f300 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.693+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/3111517023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4819f880 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.693+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/3111517023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b481088c0 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.694+0000 7f6b2ffff700 1 -- 172.21.15.179:0/3111517023 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 91458+0+0 (secure 0 0 0) 0x7f6b4000cb10 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.694+0000 7f6b2ffff700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6b3006db10 0x7f6b3006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.694+0000 7f6b2ffff700 1 -- 172.21.15.179:0/3111517023 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f6b40006e40 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.694+0000 7f6b46ffd700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6b3006db10 0x7f6b3006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.695+0000 7f6b46ffd700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6b3006db10 0x7f6b3006ffd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6b38006010 tx=0x7f6b38009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.700+0000 7f6b2ffff700 1 -- 172.21.15.179:0/3111517023 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b400565f0 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.975+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/3111517023 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f6b4810ccd0 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.980+0000 7f6b2ffff700 1 -- 172.21.15.179:0/3111517023 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f6b40019020 con 0x7f6b48103880 2024-02-14T17:45:27.225 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6b3006db10 msgr2=0x7f6b3006ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6b3006db10 0x7f6b3006ffd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6b38006010 tx=0x7f6b38009f10 comp rx=0 tx=0).stop 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/3111517023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b48103880 msgr2=0x7f6b4819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b48103880 0x7f6b4819e780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000b970 tx=0x7f6b4000bd30 comp rx=0 tx=0).stop 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/3111517023 shutdown_connections 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f6b3006db10 0x7f6b3006ffd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b48103880 0x7f6b4819e780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b48101ce0 0x7f6b481971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 --2- 172.21.15.179:0/3111517023 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f6b48102ee0 0x7f6b48197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/3111517023 >> 172.21.15.179:0/3111517023 conn(0x7f6b480fd2c0 msgr2=0x7f6b480fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/3111517023 shutdown_connections 2024-02-14T17:45:27.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-02-14T17:45:26.982+0000 7f6b4d3f1700 1 -- 172.21.15.179:0/3111517023 wait complete. 2024-02-14T17:45:27.226 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:27 smithi179 ceph-mon[33633]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-02-14T17:45:27.227 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.179:0/3111517023' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:27.227 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:27.227 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:45:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:27 smithi100 ceph-mon[30138]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-02-14T17:45:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.179:0/3111517023' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:45:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:27 smithi100 ceph-mon[39557]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 68 B/s, 1 objects/s recovering 2024-02-14T17:45:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.179:0/3111517023' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-14T17:45:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-14T17:45:27.843 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-02-14T17:45:27.843 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-14T17:45:27.843 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-14T17:45:27.889 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-14T17:45:27.889 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-14T17:45:27.889 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph mgr dump --format=json 2024-02-14T17:45:28.139 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:29.176 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:29 smithi179 ceph-mon[33633]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 997 B/s rd, 0 op/s; 59 B/s, 1 objects/s recovering 2024-02-14T17:45:29.385 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:29 smithi100 ceph-mon[39557]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 997 B/s rd, 0 op/s; 59 B/s, 1 objects/s recovering 2024-02-14T17:45:29.385 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:29 smithi100 ceph-mon[30138]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 997 B/s rd, 0 op/s; 59 B/s, 1 objects/s recovering 2024-02-14T17:45:29.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.914+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/3694760088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b08100b70 msgr2=0x7f8b08100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:29.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.914+0000 7f8b0dd5e700 1 --2- 172.21.15.100:0/3694760088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b08100b70 0x7f8b08100ff0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0098f0 tx=0x7f8afc009c00 comp rx=0 tx=0).stop 2024-02-14T17:45:29.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.914+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/3694760088 shutdown_connections 2024-02-14T17:45:29.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.914+0000 7f8b0dd5e700 1 --2- 172.21.15.100:0/3694760088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b08106b80 0x7f8b08106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:29.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.914+0000 7f8b0dd5e700 1 --2- 172.21.15.100:0/3694760088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08101530 0x7f8b0810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:29.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.914+0000 7f8b0dd5e700 1 --2- 172.21.15.100:0/3694760088 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b08100b70 0x7f8b08100ff0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:29.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.914+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/3694760088 >> 172.21.15.100:0/3694760088 conn(0x7f8b08077aa0 msgr2=0x7f8b08077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:29.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.914+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/3694760088 shutdown_connections 2024-02-14T17:45:29.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.914+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/3694760088 wait complete. 2024-02-14T17:45:29.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.915+0000 7f8b0dd5e700 1 Processor -- start 2024-02-14T17:45:29.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.915+0000 7f8b0dd5e700 1 -- start start 2024-02-14T17:45:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b0dd5e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08100b70 0x7f8b08196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b0dd5e700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b08101530 0x7f8b08196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:29.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b0dd5e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b08106b80 0x7f8b0819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b0dd5e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0819b420 con 0x7f8b08100b70 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b0dd5e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0819b590 con 0x7f8b08106b80 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b0dd5e700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0819b750 con 0x7f8b08101530 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b077fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08100b70 0x7f8b08196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b06ffd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b08101530 0x7f8b08196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b077fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08100b70 0x7f8b08196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56122/0 (socket says 172.21.15.100:56122) 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b077fe700 1 -- 172.21.15.100:0/1415890638 learned_addr learned my addr 172.21.15.100:0/1415890638 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b07fff700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b08106b80 0x7f8b0819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b06ffd700 1 -- 172.21.15.100:0/1415890638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b08106b80 msgr2=0x7f8b0819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b06ffd700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b08106b80 0x7f8b0819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b06ffd700 1 -- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08100b70 msgr2=0x7f8b08196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b06ffd700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08100b70 0x7f8b08196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:29.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b06ffd700 1 -- 172.21.15.100:0/1415890638 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b00009d00 con 0x7f8b08101530 2024-02-14T17:45:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b077fe700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08100b70 0x7f8b08196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:45:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b07fff700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b08106b80 0x7f8b0819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T17:45:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b06ffd700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b08101530 0x7f8b08196a80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009230 tx=0x7f8afc00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b04ff9700 1 -- 172.21.15.100:0/1415890638 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc01c070 con 0x7f8b08101530 2024-02-14T17:45:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/1415890638 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc009520 con 0x7f8b08101530 2024-02-14T17:45:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b04ff9700 1 -- 172.21.15.100:0/1415890638 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f8afc00bae0 con 0x7f8b08101530 2024-02-14T17:45:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.916+0000 7f8b04ff9700 1 -- 172.21.15.100:0/1415890638 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc005720 con 0x7f8b08101530 2024-02-14T17:45:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.917+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/1415890638 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b08069920 con 0x7f8b08101530 2024-02-14T17:45:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.917+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/1415890638 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b081021f0 con 0x7f8b08101530 2024-02-14T17:45:29.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.918+0000 7f8b04ff9700 1 -- 172.21.15.100:0/1415890638 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 16) v1 ==== 91458+0+0 (secure 0 0 0) 0x7f8afc020030 con 0x7f8b08101530 2024-02-14T17:45:29.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.918+0000 7f8b04ff9700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f8af006da60 0x7f8af006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:29.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.918+0000 7f8b04ff9700 1 -- 172.21.15.100:0/1415890638 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f8afc08e700 con 0x7f8b08101530 2024-02-14T17:45:29.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.918+0000 7f8b077fe700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f8af006da60 0x7f8af006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:29.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.919+0000 7f8b077fe700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f8af006da60 0x7f8af006ff20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8af800cfd0 tx=0x7f8af800e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:29.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:29.924+0000 7f8b04ff9700 1 -- 172.21.15.100:0/1415890638 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8afc057b40 con 0x7f8b08101530 2024-02-14T17:45:30.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.170+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/1415890638 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8b081978a0 con 0x7f8b08101530 2024-02-14T17:45:30.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.174+0000 7f8b04ff9700 1 -- 172.21.15.100:0/1415890638 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+174867 (secure 0 0 0) 0x7f8afc0051d0 con 0x7f8b08101530 2024-02-14T17:45:30.178 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:45:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f8af006da60 msgr2=0x7f8af006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f8af006da60 0x7f8af006ff20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8af800cfd0 tx=0x7f8af800e040 comp rx=0 tx=0).stop 2024-02-14T17:45:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b08101530 msgr2=0x7f8b08196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b08101530 0x7f8b08196a80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009230 tx=0x7f8afc00fa00 comp rx=0 tx=0).stop 2024-02-14T17:45:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/1415890638 shutdown_connections 2024-02-14T17:45:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f8af006da60 0x7f8af006ff20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b08106b80 0x7f8b0819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b08100b70 0x7f8b08196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 --2- 172.21.15.100:0/1415890638 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f8b08101530 0x7f8b08196a80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:30.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/1415890638 >> 172.21.15.100:0/1415890638 conn(0x7f8b08077aa0 msgr2=0x7f8b08107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:30.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/1415890638 shutdown_connections 2024-02-14T17:45:30.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:30.177+0000 7f8b0dd5e700 1 -- 172.21.15.100:0/1415890638 wait complete. 2024-02-14T17:45:30.322 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:29 smithi179 podman[64321]: 2024-02-14 17:45:29.848407126 +0000 UTC m=+0.018616643 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-14T17:45:30.921 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3957561430},{"type":"v1","addr":"172.21.15.100:6801","nonce":3957561430}]},"active_addr":"172.21.15.100:6801/3957561430","active_change":"2024-02-14T17:40:18.013941+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.100:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":2287998560}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1704234831}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":50507502}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1225679993}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1353904671}]}]} 2024-02-14T17:45:30.925 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-14T17:45:30.926 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-14T17:45:30.926 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T17:45:31.139 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:30 smithi179 podman[64321]: 2024-02-14 17:45:30.888132382 +0000 UTC m=+1.058341887 container create 0f6bf504887a0d3c5e4728bc6555cfc2833bc4c77c0502ce0e6b65b3b2d12bb6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:45:31.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:31 smithi100 ceph-mon[30138]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 999 B/s rd, 0 op/s; 53 B/s, 1 objects/s recovering 2024-02-14T17:45:31.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1415890638' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T17:45:31.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:31 smithi100 ceph-mon[39557]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 999 B/s rd, 0 op/s; 53 B/s, 1 objects/s recovering 2024-02-14T17:45:31.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1415890638' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T17:45:31.166 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:31.545 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-mon[33633]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 999 B/s rd, 0 op/s; 53 B/s, 1 objects/s recovering 2024-02-14T17:45:31.545 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1415890638' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-14T17:45:31.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:45:31.800 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 podman[64321]: 2024-02-14 17:45:31.732300856 +0000 UTC m=+1.902510362 container init 0f6bf504887a0d3c5e4728bc6555cfc2833bc4c77c0502ce0e6b65b3b2d12bb6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:45:31.800 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 podman[64321]: 2024-02-14 17:45:31.738587685 +0000 UTC m=+1.908797189 container start 0f6bf504887a0d3c5e4728bc6555cfc2833bc4c77c0502ce0e6b65b3b2d12bb6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:45:31.800 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.763Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-14T17:45:31.800 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.763Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-14T17:45:31.800 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.763Z caller=main.go:570 level=info host_details="(Linux 5.14.0-420.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 9 18:10:28 UTC 2024 x86_64 smithi179 (none))" 2024-02-14T17:45:31.800 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.763Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-14T17:45:31.800 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.763Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-14T17:45:31.800 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.764Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-14T17:45:31.800 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.764Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-14T17:45:31.800 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.766Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.766Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.768Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.768Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.416µs 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.768Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.768Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.768Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=32.105µs wal_replay_duration=296.507µs wbl_replay_duration=147ns total_replay_duration=416.748µs 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.769Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.769Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.769Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.785Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.947179ms db_storage=822ns remote_storage=1.252µs web_handler=367ns query_engine=597ns scrape=499.58µs scrape_sd=69.54µs notify=733ns notify_sd=1.266µs rules=15.193291ms tracing=5.831µs 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.785Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:45:31.785Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-14T17:45:31.801 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 bash[64321]: 0f6bf504887a0d3c5e4728bc6555cfc2833bc4c77c0502ce0e6b65b3b2d12bb6 2024-02-14T17:45:32.071 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:45:31 smithi179 systemd[1]: Started Ceph prometheus.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:45:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 -- 172.21.15.100:0/1991282358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc78101f80 msgr2=0x7efc78105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 --2- 172.21.15.100:0/1991282358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc78101f80 0x7efc78105810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc7400bb20 tx=0x7efc7400be30 comp rx=0 tx=0).stop 2024-02-14T17:45:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 -- 172.21.15.100:0/1991282358 shutdown_connections 2024-02-14T17:45:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 --2- 172.21.15.100:0/1991282358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc78105ef0 0x7efc7810dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 --2- 172.21.15.100:0/1991282358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc781015d0 0x7efc781019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 --2- 172.21.15.100:0/1991282358 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc78101f80 0x7efc78105810 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 -- 172.21.15.100:0/1991282358 >> 172.21.15.100:0/1991282358 conn(0x7efc78077170 msgr2=0x7efc780fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 -- 172.21.15.100:0/1991282358 shutdown_connections 2024-02-14T17:45:32.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 -- 172.21.15.100:0/1991282358 wait complete. 2024-02-14T17:45:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 Processor -- start 2024-02-14T17:45:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.955+0000 7efc8004e700 1 -- start start 2024-02-14T17:45:32.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc8004e700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc781015d0 0x7efc7819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc8004e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc78101f80 0x7efc7819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc8004e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc78105ef0 0x7efc781997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc8004e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc78199e50 con 0x7efc78105ef0 2024-02-14T17:45:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc8004e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc78199fc0 con 0x7efc78101f80 2024-02-14T17:45:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc8004e700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc7819a180 con 0x7efc781015d0 2024-02-14T17:45:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc7e5eb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc78105ef0 0x7efc781997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc7e5eb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc78105ef0 0x7efc781997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:59574/0 (socket says 172.21.15.100:59574) 2024-02-14T17:45:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc7e5eb700 1 -- 172.21.15.100:0/256411674 learned_addr learned my addr 172.21.15.100:0/256411674 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc7ddea700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc781015d0 0x7efc7819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:32.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc7d5e9700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc78101f80 0x7efc7819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:32.959 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:32.960 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:32.960 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:32.960 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:32.960 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-14T17:45:32.960 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[39557]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 45 B/s, 1 objects/s recovering 2024-02-14T17:45:32.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:32.960 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:32.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:32.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:32.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-14T17:45:32.961 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-mon[30138]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 45 B/s, 1 objects/s recovering 2024-02-14T17:45:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc7ddea700 1 -- 172.21.15.100:0/256411674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc78101f80 msgr2=0x7efc7819fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc7ddea700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc78101f80 0x7efc7819fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc7ddea700 1 -- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc78105ef0 msgr2=0x7efc781997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc7ddea700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc78105ef0 0x7efc781997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.956+0000 7efc7ddea700 1 -- 172.21.15.100:0/256411674 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc7400b800 con 0x7efc781015d0 2024-02-14T17:45:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.957+0000 7efc7ddea700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc781015d0 0x7efc7819f620 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efc6800e9f0 tx=0x7efc6800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.957+0000 7efc6effd700 1 -- 172.21.15.100:0/256411674 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6800eee0 con 0x7efc781015d0 2024-02-14T17:45:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.957+0000 7efc6effd700 1 -- 172.21.15.100:0/256411674 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7efc6800cbd0 con 0x7efc781015d0 2024-02-14T17:45:32.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.957+0000 7efc8004e700 1 -- 172.21.15.100:0/256411674 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc7819a460 con 0x7efc781015d0 2024-02-14T17:45:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.957+0000 7efc6effd700 1 -- 172.21.15.100:0/256411674 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68018680 con 0x7efc781015d0 2024-02-14T17:45:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.957+0000 7efc8004e700 1 -- 172.21.15.100:0/256411674 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc781a6650 con 0x7efc781015d0 2024-02-14T17:45:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.958+0000 7efc8004e700 1 -- 172.21.15.100:0/256411674 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc78102380 con 0x7efc781015d0 2024-02-14T17:45:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.959+0000 7efc6effd700 1 -- 172.21.15.100:0/256411674 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 17) v1 ==== 91472+0+0 (secure 0 0 0) 0x7efc68010430 con 0x7efc781015d0 2024-02-14T17:45:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.959+0000 7efc6effd700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7efc6406dab0 0x7efc6406ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.959+0000 7efc7d5e9700 1 -- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7efc6406dab0 msgr2=0x7efc6406ff70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/3957561430 2024-02-14T17:45:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.959+0000 7efc6effd700 1 -- 172.21.15.100:0/256411674 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7207+0+0 (secure 0 0 0) 0x7efc68014070 con 0x7efc781015d0 2024-02-14T17:45:32.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.959+0000 7efc7d5e9700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7efc6406dab0 0x7efc6406ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T17:45:32.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:32.964+0000 7efc6effd700 1 -- 172.21.15.100:0/256411674 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc68056f20 con 0x7efc781015d0 2024-02-14T17:45:33.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.160+0000 7efc7d5e9700 1 -- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7efc6406dab0 msgr2=0x7efc6406ff70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/3957561430 2024-02-14T17:45:33.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.160+0000 7efc7d5e9700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7efc6406dab0 0x7efc6406ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T17:45:33.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.162+0000 7efc8004e700 1 -- 172.21.15.100:0/256411674 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc7819acd0 con 0x7efc781015d0 2024-02-14T17:45:33.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.163+0000 7efc6effd700 1 -- 172.21.15.100:0/256411674 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+22800 (secure 0 0 0) 0x7efc680106e0 con 0x7efc781015d0 2024-02-14T17:45:33.165 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:45:33.166 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":71,"fsid":"d69a9998-cb5f-11ee-95b9-87774f69a715","created":"2024-02-14T17:39:23.866791+0000","modified":"2024-02-14T17:45:19.834604+0000","last_up_change":"2024-02-14T17:44:44.144566+0000","last_in_change":"2024-02-14T17:44:27.059067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T17:42:38.342681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-14T17:44:56.522264+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-14T17:44:57.161296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-14T17:44:58.981800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-14T17:45:00.927706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-14T17:45:02.992654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d822e357-3e1f-407f-a403-8b994ccc1f84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6803","nonce":2110588638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6805","nonce":2110588638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6809","nonce":2110588638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6807","nonce":2110588638}]},"public_addr":"172.21.15.100:6803/2110588638","cluster_addr":"172.21.15.100:6805/2110588638","heartbeat_back_addr":"172.21.15.100:6809/2110588638","heartbeat_front_addr":"172.21.15.100:6807/2110588638","state":["exists","up"]},{"osd":1,"uuid":"f74efd5d-9766-407c-9b93-8fc21cf42e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6811","nonce":4183142537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6813","nonce":4183142537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6817","nonce":4183142537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6815","nonce":4183142537}]},"public_addr":"172.21.15.100:6811/4183142537","cluster_addr":"172.21.15.100:6813/4183142537","heartbeat_back_addr":"172.21.15.100:6817/4183142537","heartbeat_front_addr":"172.21.15.100:6815/4183142537","state":["exists","up"]},{"osd":2,"uuid":"e476ae41-e2f9-43fa-8cca-7f766e005494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6819","nonce":1857810896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6821","nonce":1857810896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6825","nonce":1857810896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6823","nonce":1857810896}]},"public_addr":"172.21.15.100:6819/1857810896","cluster_addr":"172.21.15.100:6821/1857810896","heartbeat_back_addr":"172.21.15.100:6825/1857810896","heartbeat_front_addr":"172.21.15.100:6823/1857810896","state":["exists","up"]},{"osd":3,"uuid":"0a354e22-5aff-4f94-8764-0b3dfbca48f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6827","nonce":2018852967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6829","nonce":2018852967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6833","nonce":2018852967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6831","nonce":2018852967}]},"public_addr":"172.21.15.100:6827/2018852967","cluster_addr":"172.21.15.100:6829/2018852967","heartbeat_back_addr":"172.21.15.100:6833/2018852967","heartbeat_front_addr":"172.21.15.100:6831/2018852967","state":["exists","up"]},{"osd":4,"uuid":"1a753e42-d9e5-487a-b716-5ff9bd63af60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6801","nonce":952874533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6803","nonce":952874533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6807","nonce":952874533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6805","nonce":952874533}]},"public_addr":"172.21.15.179:6801/952874533","cluster_addr":"172.21.15.179:6803/952874533","heartbeat_back_addr":"172.21.15.179:6807/952874533","heartbeat_front_addr":"172.21.15.179:6805/952874533","state":["exists","up"]},{"osd":5,"uuid":"60fca708-b509-4b49-823b-e9c81ff48e6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6809","nonce":1513852186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6811","nonce":1513852186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6815","nonce":1513852186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6813","nonce":1513852186}]},"public_addr":"172.21.15.179:6809/1513852186","cluster_addr":"172.21.15.179:6811/1513852186","heartbeat_back_addr":"172.21.15.179:6815/1513852186","heartbeat_front_addr":"172.21.15.179:6813/1513852186","state":["exists","up"]},{"osd":6,"uuid":"1d797fe1-a6df-4e3d-a923-8ebbafa7a49b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6817","nonce":2309639257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6819","nonce":2309639257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6823","nonce":2309639257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6821","nonce":2309639257}]},"public_addr":"172.21.15.179:6817/2309639257","cluster_addr":"172.21.15.179:6819/2309639257","heartbeat_back_addr":"172.21.15.179:6823/2309639257","heartbeat_front_addr":"172.21.15.179:6821/2309639257","state":["exists","up"]},{"osd":7,"uuid":"51596ca9-64f6-45fd-87ce-3a83bd7b9c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6825","nonce":1058185483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6827","nonce":1058185483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6831","nonce":1058185483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6829","nonce":1058185483}]},"public_addr":"172.21.15.179:6825/1058185483","cluster_addr":"172.21.15.179:6827/1058185483","heartbeat_back_addr":"172.21.15.179:6831/1058185483","heartbeat_front_addr":"172.21.15.179:6829/1058185483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:41:43.603858+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:42:09.834569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:42:36.005505+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:43:02.929722+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:43:26.403581+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:43:51.680222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:44:17.365479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:44:43.597507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/4153927297":"2024-02-15T17:40:18.013833+0000","172.21.15.100:6800/1559623009":"2024-02-15T17:40:18.013833+0000","172.21.15.100:0/2717619524":"2024-02-15T17:40:18.013833+0000","172.21.15.100:0/3583073370":"2024-02-15T17:39:51.038137+0000","172.21.15.100:6801/1559623009":"2024-02-15T17:40:18.013833+0000","172.21.15.100:0/2663948233":"2024-02-15T17:39:51.038137+0000","172.21.15.100:0/2115528537":"2024-02-15T17:39:51.038137+0000","172.21.15.100:6800/2036660321":"2024-02-15T17:39:51.038137+0000","172.21.15.100:6801/2036660321":"2024-02-15T17:39:51.038137+0000","172.21.15.100:0/3950545425":"2024-02-15T17:40:18.013833+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T17:45:33.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.165+0000 7efc8004e700 1 -- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7efc6406dab0 msgr2=0x7efc6406ff70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:45:33.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.165+0000 7efc8004e700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7efc6406dab0 0x7efc6406ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:33.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.165+0000 7efc8004e700 1 -- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc781015d0 msgr2=0x7efc7819f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:33.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.165+0000 7efc8004e700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc781015d0 0x7efc7819f620 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efc6800e9f0 tx=0x7efc6800ed00 comp rx=0 tx=0).stop 2024-02-14T17:45:33.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.166+0000 7efc8004e700 1 -- 172.21.15.100:0/256411674 shutdown_connections 2024-02-14T17:45:33.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.166+0000 7efc8004e700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efc78105ef0 0x7efc781997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:33.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.166+0000 7efc8004e700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efc781015d0 0x7efc7819f620 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:33.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.166+0000 7efc8004e700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7efc6406dab0 0x7efc6406ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:33.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.166+0000 7efc8004e700 1 --2- 172.21.15.100:0/256411674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc78101f80 0x7efc7819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:33.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.166+0000 7efc8004e700 1 -- 172.21.15.100:0/256411674 >> 172.21.15.100:0/256411674 conn(0x7efc78077170 msgr2=0x7efc78103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:33.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.166+0000 7efc8004e700 1 -- 172.21.15.100:0/256411674 shutdown_connections 2024-02-14T17:45:33.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:33.166+0000 7efc8004e700 1 -- 172.21.15.100:0/256411674 wait complete. 2024-02-14T17:45:33.321 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: ignoring --setuser ceph since I am not root 2024-02-14T17:45:33.321 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: ignoring --setgroup ceph since I am not root 2024-02-14T17:45:33.321 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:32 smithi179 ceph-mgr[35738]: -- 172.21.15.179:0/355025795 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5634b005a1a0 con 0x5634b0030c00 2024-02-14T17:45:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:32 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:32 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:32 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:32 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' 2024-02-14T17:45:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:32 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-14T17:45:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:32 smithi179 ceph-mon[33633]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 45 B/s, 1 objects/s recovering 2024-02-14T17:45:33.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ignoring --setuser ceph since I am not root 2024-02-14T17:45:33.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:32 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ignoring --setgroup ceph since I am not root 2024-02-14T17:45:33.821 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:33 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:33.475+0000 7f9e5e92a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:45:33.821 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:33 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:33.569+0000 7f9e5e92a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:45:33.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:33 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:33.486+0000 7efc65b96200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-14T17:45:33.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:33 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:33.588+0000 7efc65b96200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-14T17:45:34.062 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-14T17:45:34.062 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-14T17:45:34.201 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:33 smithi179 ceph-mon[33633]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-14T17:45:34.201 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:33 smithi179 ceph-mon[33633]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-14T17:45:34.201 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/256411674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:45:34.249 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:33 smithi100 ceph-mon[39557]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-14T17:45:34.249 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:33 smithi100 ceph-mon[39557]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-14T17:45:34.249 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/256411674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:45:34.249 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:33 smithi100 ceph-mon[30138]: from='mgr.14152 172.21.15.100:0/484054967' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-14T17:45:34.249 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:33 smithi100 ceph-mon[30138]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-14T17:45:34.249 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/256411674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:45:34.312 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:34.504 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:34 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:34.246+0000 7efc65b96200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:45:34.504 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:34 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:34.358+0000 7efc65b96200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:45:34.505 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:34 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:34.502+0000 7efc65b96200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:45:34.530 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:34 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:34.199+0000 7f9e5e92a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-14T17:45:34.530 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:34 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:34.292+0000 7f9e5e92a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-14T17:45:34.530 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:34 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:34.433+0000 7f9e5e92a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-14T17:45:34.530 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:34 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:34.528+0000 7f9e5e92a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:45:34.760 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:34 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:34.603+0000 7efc65b96200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-14T17:45:34.760 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:34 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:34.689+0000 7efc65b96200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:45:34.821 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:34 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:34.610+0000 7f9e5e92a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-14T17:45:35.074 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:34 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:34.963+0000 7efc65b96200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:45:35.164 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:34 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:34.876+0000 7f9e5e92a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-14T17:45:35.164 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:34 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:34.984+0000 7f9e5e92a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:45:35.165 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:35 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:35.069+0000 7f9e5e92a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:45:35.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:35 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:35.073+0000 7efc65b96200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-14T17:45:35.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:35 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:35.161+0000 7efc65b96200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-14T17:45:35.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:35 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:35.257+0000 7efc65b96200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:45:35.571 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:35 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:35.162+0000 7f9e5e92a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-14T17:45:35.571 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:35 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:35.339+0000 7f9e5e92a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:45:35.746 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:35 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:35.434+0000 7efc65b96200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-14T17:45:36.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.133+0000 7f4934f30700 1 -- 172.21.15.100:0/127189397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49301026b0 msgr2=0x7f493010eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.133+0000 7f4934f30700 1 --2- 172.21.15.100:0/127189397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49301026b0 0x7f493010eb10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f492800b0a0 tx=0x7f492800b3b0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.133+0000 7f4934f30700 1 -- 172.21.15.100:0/127189397 shutdown_connections 2024-02-14T17:45:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.133+0000 7f4934f30700 1 --2- 172.21.15.100:0/127189397 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f49301026b0 0x7f493010eb10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.133+0000 7f4934f30700 1 --2- 172.21.15.100:0/127189397 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4930107dd0 0x7f49301081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.133+0000 7f4934f30700 1 --2- 172.21.15.100:0/127189397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4930101cf0 0x7f4930102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.133+0000 7f4934f30700 1 -- 172.21.15.100:0/127189397 >> 172.21.15.100:0/127189397 conn(0x7f4930078d30 msgr2=0x7f4930079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.133+0000 7f4934f30700 1 -- 172.21.15.100:0/127189397 shutdown_connections 2024-02-14T17:45:36.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.133+0000 7f4934f30700 1 -- 172.21.15.100:0/127189397 wait complete. 2024-02-14T17:45:36.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.134+0000 7f4934f30700 1 Processor -- start 2024-02-14T17:45:36.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.134+0000 7f4934f30700 1 -- start start 2024-02-14T17:45:36.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.134+0000 7f4934f30700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4930101cf0 0x7f493019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:36.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.134+0000 7f4934f30700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49301026b0 0x7f493019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.134+0000 7f4934f30700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930107dd0 0x7f4930197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.134+0000 7f4934f30700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4930197c60 con 0x7f4930107dd0 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f4934f30700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4930197da0 con 0x7f49301026b0 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f4934f30700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4930197f60 con 0x7f4930101cf0 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492e59c700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4930101cf0 0x7f493019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492e59c700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4930101cf0 0x7f493019d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:59650/0 (socket says 172.21.15.100:59650) 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492e59c700 1 -- 172.21.15.100:0/3172064862 learned_addr learned my addr 172.21.15.100:0/3172064862 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492ed9d700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930107dd0 0x7f4930197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492dd9b700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49301026b0 0x7f493019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492e59c700 1 -- 172.21.15.100:0/3172064862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49301026b0 msgr2=0x7f493019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492e59c700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49301026b0 0x7f493019d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492e59c700 1 -- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930107dd0 msgr2=0x7f4930197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492e59c700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930107dd0 0x7f4930197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492e59c700 1 -- 172.21.15.100:0/3172064862 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928009d00 con 0x7f4930101cf0 2024-02-14T17:45:36.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492dd9b700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49301026b0 0x7f493019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T17:45:36.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492ed9d700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930107dd0 0x7f4930197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:45:36.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f492e59c700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4930101cf0 0x7f493019d440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f492000e9a0 tx=0x7f492000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:36.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f491f7fe700 1 -- 172.21.15.100:0/3172064862 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920009790 con 0x7f4930101cf0 2024-02-14T17:45:36.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f4934f30700 1 -- 172.21.15.100:0/3172064862 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4930198240 con 0x7f4930101cf0 2024-02-14T17:45:36.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f491f7fe700 1 -- 172.21.15.100:0/3172064862 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f492000cbd0 con 0x7f4930101cf0 2024-02-14T17:45:36.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f491f7fe700 1 -- 172.21.15.100:0/3172064862 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920018420 con 0x7f4930101cf0 2024-02-14T17:45:36.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.135+0000 7f4934f30700 1 -- 172.21.15.100:0/3172064862 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49301a44e0 con 0x7f4930101cf0 2024-02-14T17:45:36.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.136+0000 7f4934f30700 1 -- 172.21.15.100:0/3172064862 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4930068930 con 0x7f4930101cf0 2024-02-14T17:45:36.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.137+0000 7f491f7fe700 1 -- 172.21.15.100:0/3172064862 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 17) v1 ==== 91472+0+0 (secure 0 0 0) 0x7f4920010430 con 0x7f4930101cf0 2024-02-14T17:45:36.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.137+0000 7f491f7fe700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f491806dab0 0x7f491806ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:36.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.137+0000 7f492dd9b700 1 -- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f491806dab0 msgr2=0x7f491806ff70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/3957561430 2024-02-14T17:45:36.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.137+0000 7f491f7fe700 1 -- 172.21.15.100:0/3172064862 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f4920014070 con 0x7f4930101cf0 2024-02-14T17:45:36.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.137+0000 7f492dd9b700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f491806dab0 0x7f491806ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-14T17:45:36.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.142+0000 7f491f7fe700 1 -- 172.21.15.100:0/3172064862 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4920052d20 con 0x7f4930101cf0 2024-02-14T17:45:36.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.338+0000 7f492dd9b700 1 -- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f491806dab0 msgr2=0x7f491806ff70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.100:6800/3957561430 2024-02-14T17:45:36.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.338+0000 7f492dd9b700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f491806dab0 0x7f491806ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-14T17:45:36.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.339+0000 7f4934f30700 1 -- 172.21.15.100:0/3172064862 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4930111190 con 0x7f4930101cf0 2024-02-14T17:45:36.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.341+0000 7f491f7fe700 1 -- 172.21.15.100:0/3172064862 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+22800 (secure 0 0 0) 0x7f4920056e60 con 0x7f4930101cf0 2024-02-14T17:45:36.343 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:45:36.343 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":71,"fsid":"d69a9998-cb5f-11ee-95b9-87774f69a715","created":"2024-02-14T17:39:23.866791+0000","modified":"2024-02-14T17:45:19.834604+0000","last_up_change":"2024-02-14T17:44:44.144566+0000","last_in_change":"2024-02-14T17:44:27.059067+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-14T17:42:38.342681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-14T17:44:56.522264+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-14T17:44:57.161296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-14T17:44:58.981800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-14T17:45:00.927706+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-14T17:45:02.992654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d822e357-3e1f-407f-a403-8b994ccc1f84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6803","nonce":2110588638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6805","nonce":2110588638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6809","nonce":2110588638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":2110588638},{"type":"v1","addr":"172.21.15.100:6807","nonce":2110588638}]},"public_addr":"172.21.15.100:6803/2110588638","cluster_addr":"172.21.15.100:6805/2110588638","heartbeat_back_addr":"172.21.15.100:6809/2110588638","heartbeat_front_addr":"172.21.15.100:6807/2110588638","state":["exists","up"]},{"osd":1,"uuid":"f74efd5d-9766-407c-9b93-8fc21cf42e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6811","nonce":4183142537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6813","nonce":4183142537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6817","nonce":4183142537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":4183142537},{"type":"v1","addr":"172.21.15.100:6815","nonce":4183142537}]},"public_addr":"172.21.15.100:6811/4183142537","cluster_addr":"172.21.15.100:6813/4183142537","heartbeat_back_addr":"172.21.15.100:6817/4183142537","heartbeat_front_addr":"172.21.15.100:6815/4183142537","state":["exists","up"]},{"osd":2,"uuid":"e476ae41-e2f9-43fa-8cca-7f766e005494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6819","nonce":1857810896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6821","nonce":1857810896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6825","nonce":1857810896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1857810896},{"type":"v1","addr":"172.21.15.100:6823","nonce":1857810896}]},"public_addr":"172.21.15.100:6819/1857810896","cluster_addr":"172.21.15.100:6821/1857810896","heartbeat_back_addr":"172.21.15.100:6825/1857810896","heartbeat_front_addr":"172.21.15.100:6823/1857810896","state":["exists","up"]},{"osd":3,"uuid":"0a354e22-5aff-4f94-8764-0b3dfbca48f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6827","nonce":2018852967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6829","nonce":2018852967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6833","nonce":2018852967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":2018852967},{"type":"v1","addr":"172.21.15.100:6831","nonce":2018852967}]},"public_addr":"172.21.15.100:6827/2018852967","cluster_addr":"172.21.15.100:6829/2018852967","heartbeat_back_addr":"172.21.15.100:6833/2018852967","heartbeat_front_addr":"172.21.15.100:6831/2018852967","state":["exists","up"]},{"osd":4,"uuid":"1a753e42-d9e5-487a-b716-5ff9bd63af60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6801","nonce":952874533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6803","nonce":952874533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6807","nonce":952874533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":952874533},{"type":"v1","addr":"172.21.15.179:6805","nonce":952874533}]},"public_addr":"172.21.15.179:6801/952874533","cluster_addr":"172.21.15.179:6803/952874533","heartbeat_back_addr":"172.21.15.179:6807/952874533","heartbeat_front_addr":"172.21.15.179:6805/952874533","state":["exists","up"]},{"osd":5,"uuid":"60fca708-b509-4b49-823b-e9c81ff48e6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6809","nonce":1513852186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6811","nonce":1513852186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6815","nonce":1513852186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1513852186},{"type":"v1","addr":"172.21.15.179:6813","nonce":1513852186}]},"public_addr":"172.21.15.179:6809/1513852186","cluster_addr":"172.21.15.179:6811/1513852186","heartbeat_back_addr":"172.21.15.179:6815/1513852186","heartbeat_front_addr":"172.21.15.179:6813/1513852186","state":["exists","up"]},{"osd":6,"uuid":"1d797fe1-a6df-4e3d-a923-8ebbafa7a49b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6817","nonce":2309639257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6819","nonce":2309639257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6823","nonce":2309639257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":2309639257},{"type":"v1","addr":"172.21.15.179:6821","nonce":2309639257}]},"public_addr":"172.21.15.179:6817/2309639257","cluster_addr":"172.21.15.179:6819/2309639257","heartbeat_back_addr":"172.21.15.179:6823/2309639257","heartbeat_front_addr":"172.21.15.179:6821/2309639257","state":["exists","up"]},{"osd":7,"uuid":"51596ca9-64f6-45fd-87ce-3a83bd7b9c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6825","nonce":1058185483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6827","nonce":1058185483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6831","nonce":1058185483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1058185483},{"type":"v1","addr":"172.21.15.179:6829","nonce":1058185483}]},"public_addr":"172.21.15.179:6825/1058185483","cluster_addr":"172.21.15.179:6827/1058185483","heartbeat_back_addr":"172.21.15.179:6831/1058185483","heartbeat_front_addr":"172.21.15.179:6829/1058185483","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:41:43.603858+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:42:09.834569+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:42:36.005505+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:43:02.929722+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:43:26.403581+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:43:51.680222+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:44:17.365479+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-14T17:44:43.597507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.100:0/4153927297":"2024-02-15T17:40:18.013833+0000","172.21.15.100:6800/1559623009":"2024-02-15T17:40:18.013833+0000","172.21.15.100:0/2717619524":"2024-02-15T17:40:18.013833+0000","172.21.15.100:0/3583073370":"2024-02-15T17:39:51.038137+0000","172.21.15.100:6801/1559623009":"2024-02-15T17:40:18.013833+0000","172.21.15.100:0/2663948233":"2024-02-15T17:39:51.038137+0000","172.21.15.100:0/2115528537":"2024-02-15T17:39:51.038137+0000","172.21.15.100:6800/2036660321":"2024-02-15T17:39:51.038137+0000","172.21.15.100:6801/2036660321":"2024-02-15T17:39:51.038137+0000","172.21.15.100:0/3950545425":"2024-02-15T17:40:18.013833+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-14T17:45:36.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 -- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f491806dab0 msgr2=0x7f491806ff70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-14T17:45:36.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f491806dab0 0x7f491806ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 -- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4930101cf0 msgr2=0x7f493019d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:36.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4930101cf0 0x7f493019d440 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f492000e9a0 tx=0x7f492000ecb0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 -- 172.21.15.100:0/3172064862 shutdown_connections 2024-02-14T17:45:36.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4930107dd0 0x7f4930197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4930101cf0 0x7f493019d440 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.100:6800/3957561430,v1:172.21.15.100:6801/3957561430] conn(0x7f491806dab0 0x7f491806ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 --2- 172.21.15.100:0/3172064862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f49301026b0 0x7f493019d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:36.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 -- 172.21.15.100:0/3172064862 >> 172.21.15.100:0/3172064862 conn(0x7f4930078d30 msgr2=0x7f49301058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:36.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 -- 172.21.15.100:0/3172064862 shutdown_connections 2024-02-14T17:45:36.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:36.343+0000 7f4934f30700 1 -- 172.21.15.100:0/3172064862 wait complete. 2024-02-14T17:45:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3172064862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:45:36.571 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:36 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:36.292+0000 7f9e5e92a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:45:36.572 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:36 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:36.423+0000 7f9e5e92a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:45:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3172064862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:45:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3172064862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-14T17:45:36.651 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:36 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:36.432+0000 7efc65b96200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-14T17:45:36.651 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:36 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:36.566+0000 7efc65b96200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-14T17:45:36.948 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:36 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:36.906+0000 7efc65b96200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:45:36.977 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-14T17:45:36.978 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-14T17:45:36.978 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-14T17:45:36.979 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-14T17:45:36.979 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-14T17:45:36.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-14T17:45:36.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-14T17:45:36.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-14T17:45:37.047 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:36 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:36.761+0000 7f9e5e92a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-14T17:45:37.047 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:36 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:36.850+0000 7f9e5e92a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:45:37.321 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:37 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:37.046+0000 7f9e5e92a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:45:37.321 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:37 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:37.230+0000 7f9e5e92a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:45:37.385 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:37 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:37.015+0000 7efc65b96200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-14T17:45:37.386 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:37 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:37.383+0000 7efc65b96200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:45:37.482 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:37.534 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:37.777 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:37 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:37.514+0000 7f9e5e92a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:45:37.891 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:37 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:37.639+0000 7efc65b96200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-14T17:45:38.071 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:37 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:37.776+0000 7f9e5e92a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:45:38.071 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:38 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:38.011+0000 7f9e5e92a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:45:38.205 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:37 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:37.938+0000 7efc65b96200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-14T17:45:38.465 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:38 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:38.203+0000 7efc65b96200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-14T17:45:38.465 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:38 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:38.448+0000 7efc65b96200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-14T17:45:38.792 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:39.071 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:38 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:38.810+0000 7f9e5e92a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:45:39.594 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:39.273+0000 7efc65b96200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-14T17:45:39.697 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:39.382+0000 7f9e5e92a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:45:39.697 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:39.478+0000 7f9e5e92a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:45:39.697 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:39.695+0000 7f9e5e92a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:45:39.873 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[39557]: Standby manager daemon x restarted 2024-02-14T17:45:39.873 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[39557]: Standby manager daemon x started 2024-02-14T17:45:39.873 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[39557]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:45:39.874 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[39557]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:45:39.874 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[39557]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:45:39.874 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[39557]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:45:39.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[30138]: Standby manager daemon x restarted 2024-02-14T17:45:39.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[30138]: Standby manager daemon x started 2024-02-14T17:45:39.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[30138]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:45:39.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[30138]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:45:39.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[30138]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:45:39.874 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-mon[30138]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:45:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-mon[33633]: Standby manager daemon x restarted 2024-02-14T17:45:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-mon[33633]: Standby manager daemon x started 2024-02-14T17:45:40.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-mon[33633]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-14T17:45:40.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-mon[33633]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-14T17:45:40.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-mon[33633]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-14T17:45:40.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-mon[33633]: from='mgr.? 172.21.15.179:0/2032968334' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-14T17:45:40.072 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: [14/Feb/2024:17:45:39] ENGINE Bus STARTING 2024-02-14T17:45:40.072 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: CherryPy Checker: 2024-02-14T17:45:40.072 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: The Application mounted at '' has an empty config. 2024-02-14T17:45:40.072 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: 2024-02-14T17:45:40.072 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: [14/Feb/2024:17:45:39] ENGINE Serving on http://:::9283 2024-02-14T17:45:40.072 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 17:45:39 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x[35715]: [14/Feb/2024:17:45:39] ENGINE Bus STARTED 2024-02-14T17:45:40.136 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:39.871+0000 7efc65b96200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-14T17:45:40.136 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:39 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:39.976+0000 7efc65b96200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-14T17:45:40.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:40.203+0000 7efc65b96200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-14T17:45:40.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:45:40] ENGINE Bus STARTING 2024-02-14T17:45:40.771 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-14T17:45:40.772 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: Active manager daemon y restarted 2024-02-14T17:45:40.772 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: Activating manager daemon y 2024-02-14T17:45:40.772 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: CherryPy Checker: 2024-02-14T17:45:40.772 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: The Application mounted at '' has an empty config. 2024-02-14T17:45:40.772 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T17:45:40.772 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:45:40] ENGINE Serving on http://:::9283 2024-02-14T17:45:40.772 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:45:40] ENGINE Bus STARTED 2024-02-14T17:45:40.772 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:45:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:45:41.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: osdmap e72: 8 total, 8 up, 8 in 2024-02-14T17:45:41.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: mgrmap e19: y(active, starting, since 0.00752705s), standbys: x 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:45:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: Manager daemon y is now available 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:45:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:45:41.074 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:40 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:45:41.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-14T17:45:41.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: Active manager daemon y restarted 2024-02-14T17:45:41.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: Activating manager daemon y 2024-02-14T17:45:41.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: osdmap e72: 8 total, 8 up, 8 in 2024-02-14T17:45:41.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: mgrmap e19: y(active, starting, since 0.00752705s), standbys: x 2024-02-14T17:45:41.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:45:41.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:45:41.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:45:41.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:45:41.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: Manager daemon y is now available 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:41.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:41.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:41.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:41.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:45:41.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:45:41.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:45:41.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:45:41.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-14T17:45:41.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: Active manager daemon y restarted 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: Activating manager daemon y 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: osdmap e72: 8 total, 8 up, 8 in 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: mgrmap e19: y(active, starting, since 0.00752705s), standbys: x 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-14T17:45:41.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-14T17:45:41.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-14T17:45:41.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-14T17:45:41.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-14T17:45:41.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-14T17:45:41.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: Manager daemon y is now available 2024-02-14T17:45:41.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:41.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:45:41.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:41.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:41.156 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:45:41.157 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-14T17:45:41.157 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:45:41.157 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:40 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-14T17:45:41.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:45:42.519 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:42 smithi100 ceph-mon[30138]: mgrmap e20: y(active, since 1.01103s), standbys: x 2024-02-14T17:45:42.519 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:42 smithi100 ceph-mon[30138]: [14/Feb/2024:17:45:41] ENGINE Bus STARTING 2024-02-14T17:45:42.519 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:42 smithi100 ceph-mon[39557]: mgrmap e20: y(active, since 1.01103s), standbys: x 2024-02-14T17:45:42.519 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:42 smithi100 ceph-mon[39557]: [14/Feb/2024:17:45:41] ENGINE Bus STARTING 2024-02-14T17:45:42.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:42 smithi179 ceph-mon[33633]: mgrmap e20: y(active, since 1.01103s), standbys: x 2024-02-14T17:45:42.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:42 smithi179 ceph-mon[33633]: [14/Feb/2024:17:45:41] ENGINE Bus STARTING 2024-02-14T17:45:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:43 smithi179 ceph-mon[33633]: [14/Feb/2024:17:45:42] ENGINE Serving on https://172.21.15.100:7150 2024-02-14T17:45:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:43 smithi179 ceph-mon[33633]: [14/Feb/2024:17:45:42] ENGINE Serving on http://172.21.15.100:8765 2024-02-14T17:45:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:43 smithi179 ceph-mon[33633]: [14/Feb/2024:17:45:42] ENGINE Bus STARTED 2024-02-14T17:45:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:43 smithi179 ceph-mon[33633]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:45:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:43 smithi100 ceph-mon[30138]: [14/Feb/2024:17:45:42] ENGINE Serving on https://172.21.15.100:7150 2024-02-14T17:45:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:43 smithi100 ceph-mon[30138]: [14/Feb/2024:17:45:42] ENGINE Serving on http://172.21.15.100:8765 2024-02-14T17:45:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:43 smithi100 ceph-mon[30138]: [14/Feb/2024:17:45:42] ENGINE Bus STARTED 2024-02-14T17:45:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:43 smithi100 ceph-mon[30138]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:45:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:43 smithi100 ceph-mon[39557]: [14/Feb/2024:17:45:42] ENGINE Serving on https://172.21.15.100:7150 2024-02-14T17:45:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:43 smithi100 ceph-mon[39557]: [14/Feb/2024:17:45:42] ENGINE Serving on http://172.21.15.100:8765 2024-02-14T17:45:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:43 smithi100 ceph-mon[39557]: [14/Feb/2024:17:45:42] ENGINE Bus STARTED 2024-02-14T17:45:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:43 smithi100 ceph-mon[39557]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:45:43.766 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:44.381 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:44 smithi100 ceph-mon[39557]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-14T17:45:44.381 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:44 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:44.381 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:44 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:44.381 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:44 smithi100 ceph-mon[30138]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-14T17:45:44.381 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:44 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:44.381 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:44 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:44 smithi179 ceph-mon[33633]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-14T17:45:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:44 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:44 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.486+0000 7efda63d5700 1 -- 172.21.15.100:0/1855089240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda0100530 msgr2=0x7efda0100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.486+0000 7efda63d5700 1 --2- 172.21.15.100:0/1855089240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda0100530 0x7efda0100910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd900098f0 tx=0x7efd90009c00 comp rx=0 tx=0).stop 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.486+0000 7efda63d5700 1 -- 172.21.15.100:0/1855089240 shutdown_connections 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.486+0000 7efda63d5700 1 --2- 172.21.15.100:0/1855089240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda0105610 0x7efda010d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.486+0000 7efda63d5700 1 --2- 172.21.15.100:0/1855089240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda0100530 0x7efda0100910 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.486+0000 7efda63d5700 1 --2- 172.21.15.100:0/1855089240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efda0100ee0 0x7efda0104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.486+0000 7efda63d5700 1 -- 172.21.15.100:0/1855089240 >> 172.21.15.100:0/1855089240 conn(0x7efda0077170 msgr2=0x7efda00fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efda63d5700 1 -- 172.21.15.100:0/1855089240 shutdown_connections 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efda63d5700 1 -- 172.21.15.100:0/1855089240 wait complete. 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efda63d5700 1 Processor -- start 2024-02-14T17:45:44.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efda63d5700 1 -- start start 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efda63d5700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda0100530 0x7efda019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efda63d5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda0100ee0 0x7efda019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efda63d5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efda0105610 0x7efda0199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efda63d5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda0199e40 con 0x7efda0100ee0 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efda63d5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda0199fb0 con 0x7efda0105610 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efda63d5700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda019a170 con 0x7efda0100530 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efd9ffff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda0100530 0x7efda019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efd97fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda0100ee0 0x7efda019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efd97fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda0100ee0 0x7efda019fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:50998/0 (socket says 172.21.15.100:50998) 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.487+0000 7efd97fff700 1 -- 172.21.15.100:0/2338822828 learned_addr learned my addr 172.21.15.100:0/2338822828 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efda4972700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efda0105610 0x7efda0199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efd97fff700 1 -- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda0100530 msgr2=0x7efda019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efd97fff700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7efda0100530 0x7efda019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efd97fff700 1 -- 172.21.15.100:0/2338822828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efda0105610 msgr2=0x7efda0199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efd97fff700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efda0105610 0x7efda0199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efd97fff700 1 -- 172.21.15.100:0/2338822828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd98009d00 con 0x7efda0100ee0 2024-02-14T17:45:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efd97fff700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda0100ee0 0x7efda019fb50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd8800e970 tx=0x7efd8800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efd9dffb700 1 -- 172.21.15.100:0/2338822828 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88009790 con 0x7efda0100ee0 2024-02-14T17:45:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd90009520 con 0x7efda0100ee0 2024-02-14T17:45:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efd9dffb700 1 -- 172.21.15.100:0/2338822828 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7efd8800c7e0 con 0x7efda0100ee0 2024-02-14T17:45:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efd9dffb700 1 -- 172.21.15.100:0/2338822828 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd880053c0 con 0x7efda0100ee0 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.488+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda01a6470 con 0x7efda0100ee0 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.489+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efda010fab0 con 0x7efda0100ee0 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.490+0000 7efd9dffb700 1 -- 172.21.15.100:0/2338822828 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7efd88005520 con 0x7efda0100ee0 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.491+0000 7efd9dffb700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7efd8c06dac0 0x7efd8c06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.491+0000 7efd9dffb700 1 -- 172.21.15.100:0/2338822828 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7efd88014070 con 0x7efda0100ee0 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.491+0000 7efd9ffff700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7efd8c06dac0 0x7efd8c06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.491+0000 7efd9dffb700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638] conn(0x7efd8c077950 0x7efd8c079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.491+0000 7efd9dffb700 1 -- 172.21.15.100:0/2338822828 --> [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd8c07a4c0 con 0x7efd8c077950 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.491+0000 7efd9dffb700 1 -- 172.21.15.100:0/2338822828 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd8808e4b0 con 0x7efda0100ee0 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.491+0000 7efda4972700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638] conn(0x7efd8c077950 0x7efd8c079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.491+0000 7efd9ffff700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7efd8c06dac0 0x7efd8c06ff80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efd90009fd0 tx=0x7efd90019040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.491+0000 7efda4972700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638] conn(0x7efd8c077950 0x7efd8c079df0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.492+0000 7efd9dffb700 1 -- 172.21.15.100:0/2338822828 <== osd.0 v2:172.21.15.100:6802/2110588638 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efd8c07a4c0 con 0x7efd8c077950 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.508+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 --> [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efda00fe7e0 con 0x7efd8c077950 2024-02-14T17:45:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efd9dffb700 1 -- 172.21.15.100:0/2338822828 <== osd.0 v2:172.21.15.100:6802/2110588638 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efda00fe7e0 con 0x7efd8c077950 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638] conn(0x7efd8c077950 msgr2=0x7efd8c079df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda63d5700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:6802/2110588638,v1:172.21.15.100:6803/2110588638] conn(0x7efd8c077950 0x7efd8c079df0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7efd8c06dac0 msgr2=0x7efd8c06ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda63d5700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7efd8c06dac0 0x7efd8c06ff80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efd90009fd0 tx=0x7efd90019040 comp rx=0 tx=0).stop 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda0100ee0 msgr2=0x7efda019fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda63d5700 1 --2- 172.21.15.100:0/2338822828 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efda0100ee0 0x7efda019fb50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd8800e970 tx=0x7efd8800ec80 comp rx=0 tx=0).stop 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda4972700 1 -- 172.21.15.100:0/2338822828 reap_dead start 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 shutdown_connections 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 >> 172.21.15.100:0/2338822828 conn(0x7efda0077170 msgr2=0x7efda01045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 shutdown_connections 2024-02-14T17:45:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:44.509+0000 7efda63d5700 1 -- 172.21.15.100:0/2338822828 wait complete. 2024-02-14T17:45:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:45:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-14T17:45:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi179 to 3760M 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:45.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi179 to 3760M 2024-02-14T17:45:45.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:45.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi179 to 3760M 2024-02-14T17:45:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:45:45.887 INFO:teuthology.orchestra.run.smithi100.stdout:34359738417 2024-02-14T17:45:45.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-14T17:45:46.109 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.089+0000 7f539244b700 1 -- 172.21.15.100:0/2942364382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f538c100590 msgr2=0x7f538c100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.089+0000 7f539244b700 1 --2- 172.21.15.100:0/2942364382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f538c100590 0x7f538c100970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f537c0098f0 tx=0x7f537c009c00 comp rx=0 tx=0).stop 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.089+0000 7f539244b700 1 -- 172.21.15.100:0/2942364382 shutdown_connections 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.089+0000 7f539244b700 1 --2- 172.21.15.100:0/2942364382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f538c105670 0x7f538c10d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.089+0000 7f539244b700 1 --2- 172.21.15.100:0/2942364382 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f538c100590 0x7f538c100970 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.089+0000 7f539244b700 1 --2- 172.21.15.100:0/2942364382 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f538c100f40 0x7f538c104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.089+0000 7f539244b700 1 -- 172.21.15.100:0/2942364382 >> 172.21.15.100:0/2942364382 conn(0x7f538c077170 msgr2=0x7f538c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.089+0000 7f539244b700 1 -- 172.21.15.100:0/2942364382 shutdown_connections 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.089+0000 7f539244b700 1 -- 172.21.15.100:0/2942364382 wait complete. 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f539244b700 1 Processor -- start 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f539244b700 1 -- start start 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f539244b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f538c100590 0x7f538c195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:46.110 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f539244b700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f538c100f40 0x7f538c195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f539244b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f538c105670 0x7f538c199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f539244b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c19a500 con 0x7f538c100590 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f539244b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c19a670 con 0x7f538c105670 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f539244b700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538c19a830 con 0x7f538c100f40 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f538bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f538c100590 0x7f538c195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f538b7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f538c100f40 0x7f538c195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f538bfff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f538c100590 0x7f538c195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51022/0 (socket says 172.21.15.100:51022) 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.090+0000 7f538bfff700 1 -- 172.21.15.100:0/1737391501 learned_addr learned my addr 172.21.15.100:0/1737391501 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f538bfff700 1 -- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f538c100f40 msgr2=0x7f538c195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f538bfff700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f538c100f40 0x7f538c195bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f538bfff700 1 -- 172.21.15.100:0/1737391501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f538c105670 msgr2=0x7f538c199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f538bfff700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f538c105670 0x7f538c199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f538bfff700 1 -- 172.21.15.100:0/1737391501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c009520 con 0x7f538c100590 2024-02-14T17:45:46.111 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f538bfff700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f538c100590 0x7f538c195670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f537c011040 tx=0x7f537c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f53897fa700 1 -- 172.21.15.100:0/1737391501 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c01c070 con 0x7f538c100590 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f53897fa700 1 -- 172.21.15.100:0/1737391501 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f537c0054c0 con 0x7f538c100590 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c19aab0 con 0x7f538c100590 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f53897fa700 1 -- 172.21.15.100:0/1737391501 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c004a30 con 0x7f538c100590 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.091+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c1a2370 con 0x7f538c100590 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.092+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f538c10f330 con 0x7f538c100590 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.092+0000 7f53897fa700 1 -- 172.21.15.100:0/1737391501 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f537c004b90 con 0x7f538c100590 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.093+0000 7f53897fa700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f537406dac0 0x7f537406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.093+0000 7f53897fa700 1 -- 172.21.15.100:0/1737391501 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f537c08f840 con 0x7f538c100590 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.093+0000 7f538b7fe700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f537406dac0 0x7f537406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.093+0000 7f53897fa700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537] conn(0x7f5374077950 0x7f5374079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.093+0000 7f53897fa700 1 -- 172.21.15.100:0/1737391501 --> [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f537407a4c0 con 0x7f5374077950 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.093+0000 7f53897fa700 1 -- 172.21.15.100:0/1737391501 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f537c08fc90 con 0x7f538c100590 2024-02-14T17:45:46.112 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.093+0000 7f53909e8700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537] conn(0x7f5374077950 0x7f5374079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:46.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.093+0000 7f538b7fe700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f537406dac0 0x7f537406ff80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f538c1030f0 tx=0x7f538000d6f0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:46.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.093+0000 7f53909e8700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537] conn(0x7f5374077950 0x7f5374079df0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:46.113 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.094+0000 7f53897fa700 1 -- 172.21.15.100:0/1737391501 <== osd.1 v2:172.21.15.100:6810/4183142537 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f537407a4c0 con 0x7f5374077950 2024-02-14T17:45:46.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.112+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 --> [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f538c062300 con 0x7f5374077950 2024-02-14T17:45:46.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f53897fa700 1 -- 172.21.15.100:0/1737391501 <== osd.1 v2:172.21.15.100:6810/4183142537 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f538c062300 con 0x7f5374077950 2024-02-14T17:45:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537] conn(0x7f5374077950 msgr2=0x7f5374079df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f539244b700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:6810/4183142537,v1:172.21.15.100:6811/4183142537] conn(0x7f5374077950 0x7f5374079df0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f537406dac0 msgr2=0x7f537406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f539244b700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f537406dac0 0x7f537406ff80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f538c1030f0 tx=0x7f538000d6f0 comp rx=0 tx=0).stop 2024-02-14T17:45:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f538c100590 msgr2=0x7f538c195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f539244b700 1 --2- 172.21.15.100:0/1737391501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f538c100590 0x7f538c195670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f537c011040 tx=0x7f537c00bf20 comp rx=0 tx=0).stop 2024-02-14T17:45:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f53909e8700 1 -- 172.21.15.100:0/1737391501 reap_dead start 2024-02-14T17:45:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 shutdown_connections 2024-02-14T17:45:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 >> 172.21.15.100:0/1737391501 conn(0x7f538c077170 msgr2=0x7f538c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:46.115 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 shutdown_connections 2024-02-14T17:45:46.116 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:46.113+0000 7f539244b700 1 -- 172.21.15.100:0/1737391501 wait complete. 2024-02-14T17:45:47.624 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:47 smithi179 ceph-mon[33633]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:45:47.633 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:47 smithi100 ceph-mon[30138]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:45:47.633 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:47 smithi100 ceph-mon[39557]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:45:47.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.631+0000 7f90a2bd5700 1 -- 172.21.15.100:0/804154889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f909c075cb0 msgr2=0x7f909c073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:47.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.631+0000 7f90a2bd5700 1 --2- 172.21.15.100:0/804154889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f909c075cb0 0x7f909c073ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9090009990 tx=0x7f9090009ca0 comp rx=0 tx=0).stop 2024-02-14T17:45:47.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.631+0000 7f90a2bd5700 1 -- 172.21.15.100:0/804154889 shutdown_connections 2024-02-14T17:45:47.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.631+0000 7f90a2bd5700 1 --2- 172.21.15.100:0/804154889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f909c112930 0x7f909c114d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:47.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.631+0000 7f90a2bd5700 1 --2- 172.21.15.100:0/804154889 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f909c075cb0 0x7f909c073ea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.631+0000 7f90a2bd5700 1 --2- 172.21.15.100:0/804154889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f909c0743e0 0x7f909c074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.631+0000 7f90a2bd5700 1 -- 172.21.15.100:0/804154889 >> 172.21.15.100:0/804154889 conn(0x7f909c078dc0 msgr2=0x7f909c0791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.631+0000 7f90a2bd5700 1 -- 172.21.15.100:0/804154889 shutdown_connections 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.631+0000 7f90a2bd5700 1 -- 172.21.15.100:0/804154889 wait complete. 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f90a2bd5700 1 Processor -- start 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f90a2bd5700 1 -- start start 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f90a2bd5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f909c0743e0 0x7f909c19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f90a2bd5700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f909c075cb0 0x7f909c19e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f90a2bd5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f909c112930 0x7f909c1a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f90a2bd5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c1a2c90 con 0x7f909c112930 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f90a2bd5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c1a2e00 con 0x7f909c0743e0 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f90a2bd5700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c1a2fc0 con 0x7f909c075cb0 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f909bfff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f909c075cb0 0x7f909c19e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f90a1172700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f909c112930 0x7f909c1a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f909bfff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f909c075cb0 0x7f909c19e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:59284/0 (socket says 172.21.15.100:59284) 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.632+0000 7f909bfff700 1 -- 172.21.15.100:0/3047454695 learned_addr learned my addr 172.21.15.100:0/3047454695 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f90a0971700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f909c0743e0 0x7f909c19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f909bfff700 1 -- 172.21.15.100:0/3047454695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f909c0743e0 msgr2=0x7f909c19ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f909bfff700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f909c0743e0 0x7f909c19ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f909bfff700 1 -- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f909c112930 msgr2=0x7f909c1a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f909bfff700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f909c112930 0x7f909c1a2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f909bfff700 1 -- 172.21.15.100:0/3047454695 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9088009520 con 0x7f909c075cb0 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f90a1172700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f909c112930 0x7f909c1a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f909bfff700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f909c075cb0 0x7f909c19e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f908800eb30 tx=0x7f908800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f9099ffb700 1 -- 172.21.15.100:0/3047454695 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088009880 con 0x7f909c075cb0 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90900095f0 con 0x7f909c075cb0 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f9099ffb700 1 -- 172.21.15.100:0/3047454695 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f90880044a0 con 0x7f909c075cb0 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f9099ffb700 1 -- 172.21.15.100:0/3047454695 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9088003ca0 con 0x7f909c075cb0 2024-02-14T17:45:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.633+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c1aa930 con 0x7f909c075cb0 2024-02-14T17:45:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.634+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f909c117ae0 con 0x7f909c075cb0 2024-02-14T17:45:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.635+0000 7f9099ffb700 1 -- 172.21.15.100:0/3047454695 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f9088003e00 con 0x7f909c075cb0 2024-02-14T17:45:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.635+0000 7f9099ffb700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f908c06dac0 0x7f908c06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.635+0000 7f9099ffb700 1 -- 172.21.15.100:0/3047454695 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f9088014070 con 0x7f909c075cb0 2024-02-14T17:45:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.635+0000 7f90a0971700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f908c06dac0 0x7f908c06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.635+0000 7f9099ffb700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483] conn(0x7f908c077950 0x7f908c079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.635+0000 7f9099ffb700 1 -- 172.21.15.100:0/3047454695 --> [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f908c07a4c0 con 0x7f908c077950 2024-02-14T17:45:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.635+0000 7f9099ffb700 1 -- 172.21.15.100:0/3047454695 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90880922c0 con 0x7f909c075cb0 2024-02-14T17:45:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.636+0000 7f90a1172700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483] conn(0x7f908c077950 0x7f908c079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.636+0000 7f90a0971700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f908c06dac0 0x7f908c06ff80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f909000b2e0 tx=0x7f9090005fd0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.637+0000 7f90a1172700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483] conn(0x7f908c077950 0x7f908c079df0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.638+0000 7f9099ffb700 1 -- 172.21.15.100:0/3047454695 <== osd.7 v2:172.21.15.179:6824/1058185483 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f908c07a4c0 con 0x7f908c077950 2024-02-14T17:45:47.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.656+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 --> [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f909c062300 con 0x7f908c077950 2024-02-14T17:45:47.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.656+0000 7f9099ffb700 1 -- 172.21.15.100:0/3047454695 <== osd.7 v2:172.21.15.179:6824/1058185483 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f909c062300 con 0x7f908c077950 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 >> [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483] conn(0x7f908c077950 msgr2=0x7f908c079df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a2bd5700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.179:6824/1058185483,v1:172.21.15.179:6825/1058185483] conn(0x7f908c077950 0x7f908c079df0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f908c06dac0 msgr2=0x7f908c06ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a2bd5700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f908c06dac0 0x7f908c06ff80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f909000b2e0 tx=0x7f9090005fd0 comp rx=0 tx=0).stop 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f909c075cb0 msgr2=0x7f909c19e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a2bd5700 1 --2- 172.21.15.100:0/3047454695 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f909c075cb0 0x7f909c19e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f908800eb30 tx=0x7f908800ee40 comp rx=0 tx=0).stop 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a1172700 1 -- 172.21.15.100:0/3047454695 reap_dead start 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 shutdown_connections 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 >> 172.21.15.100:0/3047454695 conn(0x7f909c078dc0 msgr2=0x7f909c1111b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 shutdown_connections 2024-02-14T17:45:47.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:47.657+0000 7f90a2bd5700 1 -- 172.21.15.100:0/3047454695 wait complete. 2024-02-14T17:45:48.686 INFO:teuthology.orchestra.run.smithi100.stdout:231928233998 2024-02-14T17:45:48.687 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.123+0000 7f12bd2ab700 1 -- 172.21.15.100:0/2662132706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b8101530 msgr2=0x7f12b810d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.123+0000 7f12bd2ab700 1 --2- 172.21.15.100:0/2662132706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b8101530 0x7f12b810d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12b000b0a0 tx=0x7f12b000b3b0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.123+0000 7f12bd2ab700 1 -- 172.21.15.100:0/2662132706 shutdown_connections 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.123+0000 7f12bd2ab700 1 --2- 172.21.15.100:0/2662132706 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b8101530 0x7f12b810d9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.123+0000 7f12bd2ab700 1 --2- 172.21.15.100:0/2662132706 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12b8106b80 0x7f12b8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.123+0000 7f12bd2ab700 1 --2- 172.21.15.100:0/2662132706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b8100b70 0x7f12b8100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.123+0000 7f12bd2ab700 1 -- 172.21.15.100:0/2662132706 >> 172.21.15.100:0/2662132706 conn(0x7f12b8077aa0 msgr2=0x7f12b8077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.123+0000 7f12bd2ab700 1 -- 172.21.15.100:0/2662132706 shutdown_connections 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.123+0000 7f12bd2ab700 1 -- 172.21.15.100:0/2662132706 wait complete. 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12bd2ab700 1 Processor -- start 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12bd2ab700 1 -- start start 2024-02-14T17:45:49.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12bd2ab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b8100b70 0x7f12b819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12bd2ab700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12b8101530 0x7f12b819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12bd2ab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b8106b80 0x7f12b8197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12bd2ab700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8197bf0 con 0x7f12b8106b80 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12bd2ab700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8197d30 con 0x7f12b8100b70 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12bd2ab700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8197ef0 con 0x7f12b8101530 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12b77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b8106b80 0x7f12b8197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12b77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b8106b80 0x7f12b8197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51062/0 (socket says 172.21.15.100:51062) 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12b77fe700 1 -- 172.21.15.100:0/3670019130 learned_addr learned my addr 172.21.15.100:0/3670019130 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12affff700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12b8101530 0x7f12b819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12b6ffd700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b8100b70 0x7f12b819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12b77fe700 1 -- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12b8101530 msgr2=0x7f12b819d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12b77fe700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12b8101530 0x7f12b819d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12b77fe700 1 -- 172.21.15.100:0/3670019130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b8100b70 msgr2=0x7f12b819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12b77fe700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b8100b70 0x7f12b819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12b77fe700 1 -- 172.21.15.100:0/3670019130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a8009520 con 0x7f12b8106b80 2024-02-14T17:45:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.124+0000 7f12affff700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f12b8101530 0x7f12b819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:45:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.125+0000 7f12b77fe700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b8106b80 0x7f12b8197590 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f12b000b070 tx=0x7f12b0007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.125+0000 7f12b4ff9700 1 -- 172.21.15.100:0/3670019130 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b000e040 con 0x7f12b8106b80 2024-02-14T17:45:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.125+0000 7f12b4ff9700 1 -- 172.21.15.100:0/3670019130 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f12b001d470 con 0x7f12b8106b80 2024-02-14T17:45:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.125+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b0009d00 con 0x7f12b8106b80 2024-02-14T17:45:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.125+0000 7f12b4ff9700 1 -- 172.21.15.100:0/3670019130 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b00052a0 con 0x7f12b8106b80 2024-02-14T17:45:49.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.125+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b81a42a0 con 0x7f12b8106b80 2024-02-14T17:45:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.126+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f12b8110300 con 0x7f12b8106b80 2024-02-14T17:45:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.126+0000 7f12b4ff9700 1 -- 172.21.15.100:0/3670019130 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f12b0004a50 con 0x7f12b8106b80 2024-02-14T17:45:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.127+0000 7f12b4ff9700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f12a406dac0 0x7f12a406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:49.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.127+0000 7f12b4ff9700 1 -- 172.21.15.100:0/3670019130 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f12b008ed90 con 0x7f12b8106b80 2024-02-14T17:45:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.127+0000 7f12b6ffd700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f12a406dac0 0x7f12a406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.127+0000 7f12b4ff9700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896] conn(0x7f12a40779d0 0x7f12a4079e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.127+0000 7f12b4ff9700 1 -- 172.21.15.100:0/3670019130 --> [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12a407a540 con 0x7f12a40779d0 2024-02-14T17:45:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.127+0000 7f12b4ff9700 1 -- 172.21.15.100:0/3670019130 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b0091450 con 0x7f12b8106b80 2024-02-14T17:45:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.127+0000 7f12affff700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896] conn(0x7f12a40779d0 0x7f12a4079e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.127+0000 7f12b6ffd700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f12a406dac0 0x7f12a406ff80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009df0 tx=0x7f12a8009260 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.127+0000 7f12affff700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896] conn(0x7f12a40779d0 0x7f12a4079e70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:49.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.128+0000 7f12b4ff9700 1 -- 172.21.15.100:0/3670019130 <== osd.2 v2:172.21.15.100:6818/1857810896 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f12a407a540 con 0x7f12a40779d0 2024-02-14T17:45:49.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.144+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 --> [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f12b81000e0 con 0x7f12a40779d0 2024-02-14T17:45:49.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.144+0000 7f12b4ff9700 1 -- 172.21.15.100:0/3670019130 <== osd.2 v2:172.21.15.100:6818/1857810896 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f12b81000e0 con 0x7f12a40779d0 2024-02-14T17:45:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.144+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896] conn(0x7f12a40779d0 msgr2=0x7f12a4079e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.144+0000 7f12bd2ab700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:6818/1857810896,v1:172.21.15.100:6819/1857810896] conn(0x7f12a40779d0 0x7f12a4079e70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.145+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f12a406dac0 msgr2=0x7f12a406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.145+0000 7f12bd2ab700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f12a406dac0 0x7f12a406ff80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009df0 tx=0x7f12a8009260 comp rx=0 tx=0).stop 2024-02-14T17:45:49.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.145+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b8106b80 msgr2=0x7f12b8197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.145+0000 7f12bd2ab700 1 --2- 172.21.15.100:0/3670019130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12b8106b80 0x7f12b8197590 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f12b000b070 tx=0x7f12b0007930 comp rx=0 tx=0).stop 2024-02-14T17:45:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.145+0000 7f12b77fe700 1 -- 172.21.15.100:0/3670019130 reap_dead start 2024-02-14T17:45:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.145+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 shutdown_connections 2024-02-14T17:45:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.145+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 >> 172.21.15.100:0/3670019130 conn(0x7f12b8077aa0 msgr2=0x7f12b80ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.145+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 shutdown_connections 2024-02-14T17:45:49.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.145+0000 7f12bd2ab700 1 -- 172.21.15.100:0/3670019130 wait complete. 2024-02-14T17:45:49.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.316+0000 7f956149f700 1 -- 172.21.15.100:0/3253798296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f955c101d20 msgr2=0x7f955c1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.316+0000 7f956149f700 1 --2- 172.21.15.100:0/3253798296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f955c101d20 0x7f955c1021a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9550009990 tx=0x7f9550009ca0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.316+0000 7f956149f700 1 -- 172.21.15.100:0/3253798296 shutdown_connections 2024-02-14T17:45:49.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.316+0000 7f956149f700 1 --2- 172.21.15.100:0/3253798296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f955c107d90 0x7f955c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.316+0000 7f956149f700 1 --2- 172.21.15.100:0/3253798296 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f955c1026e0 0x7f955c10ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.316+0000 7f956149f700 1 --2- 172.21.15.100:0/3253798296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f955c101d20 0x7f955c1021a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.316+0000 7f956149f700 1 -- 172.21.15.100:0/3253798296 >> 172.21.15.100:0/3253798296 conn(0x7f955c078d40 msgr2=0x7f955c079150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:49.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.316+0000 7f956149f700 1 -- 172.21.15.100:0/3253798296 shutdown_connections 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.316+0000 7f956149f700 1 -- 172.21.15.100:0/3253798296 wait complete. 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f956149f700 1 Processor -- start 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f956149f700 1 -- start start 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f956149f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f955c101d20 0x7f955c073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f956149f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f955c1026e0 0x7f955c0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f956149f700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f955c107d90 0x7f955c0749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f956149f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1a5bd0 con 0x7f955c101d20 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f956149f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1a5d40 con 0x7f955c1026e0 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f956149f700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1a5f00 con 0x7f955c107d90 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f955affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f955c101d20 0x7f955c073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:49.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f955affd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f955c101d20 0x7f955c073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51082/0 (socket says 172.21.15.100:51082) 2024-02-14T17:45:49.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f955b7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f955c107d90 0x7f955c0749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:49.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.317+0000 7f955affd700 1 -- 172.21.15.100:0/3258791066 learned_addr learned my addr 172.21.15.100:0/3258791066 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:49.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f955a7fc700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f955c1026e0 0x7f955c0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:49.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f955affd700 1 -- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f955c107d90 msgr2=0x7f955c0749f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f955affd700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f955c107d90 0x7f955c0749f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f955affd700 1 -- 172.21.15.100:0/3258791066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f955c1026e0 msgr2=0x7f955c0744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f955affd700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f955c1026e0 0x7f955c0744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f955affd700 1 -- 172.21.15.100:0/3258791066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9554009c40 con 0x7f955c101d20 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f955b7fe700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f955c107d90 0x7f955c0749f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f955affd700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f955c101d20 0x7f955c073f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f954c00d950 tx=0x7f954c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f9543fff700 1 -- 172.21.15.100:0/3258791066 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c009930 con 0x7f955c101d20 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95500095f0 con 0x7f955c101d20 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f9543fff700 1 -- 172.21.15.100:0/3258791066 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f954c010460 con 0x7f955c101d20 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f9543fff700 1 -- 172.21.15.100:0/3258791066 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c00f620 con 0x7f955c101d20 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.318+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c1a6570 con 0x7f955c101d20 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.319+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f955c111550 con 0x7f955c101d20 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.320+0000 7f9543fff700 1 -- 172.21.15.100:0/3258791066 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f954c009a90 con 0x7f955c101d20 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.320+0000 7f9543fff700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f954406dac0 0x7f954406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.320+0000 7f9543fff700 1 -- 172.21.15.100:0/3258791066 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f954c08dbc0 con 0x7f955c101d20 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.320+0000 7f955a7fc700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f954406dac0 0x7f954406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:49.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.321+0000 7f9543fff700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186] conn(0x7f95440778d0 0x7f9544079d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.321+0000 7f9543fff700 1 -- 172.21.15.100:0/3258791066 --> [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f954407a440 con 0x7f95440778d0 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.321+0000 7f9543fff700 1 -- 172.21.15.100:0/3258791066 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f954c08e010 con 0x7f955c101d20 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.321+0000 7f955a7fc700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f954406dac0 0x7f954406ff80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f955c075a00 tx=0x7f9550005940 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.321+0000 7f955b7fe700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186] conn(0x7f95440778d0 0x7f9544079d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.321+0000 7f955b7fe700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186] conn(0x7f95440778d0 0x7f9544079d70 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.322+0000 7f9543fff700 1 -- 172.21.15.100:0/3258791066 <== osd.5 v2:172.21.15.179:6808/1513852186 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f954407a440 con 0x7f95440778d0 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.338+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 --> [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f955c062300 con 0x7f95440778d0 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f9543fff700 1 -- 172.21.15.100:0/3258791066 <== osd.5 v2:172.21.15.179:6808/1513852186 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f955c062300 con 0x7f95440778d0 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 >> [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186] conn(0x7f95440778d0 msgr2=0x7f9544079d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f956149f700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.179:6808/1513852186,v1:172.21.15.179:6809/1513852186] conn(0x7f95440778d0 0x7f9544079d70 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f954406dac0 msgr2=0x7f954406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f956149f700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f954406dac0 0x7f954406ff80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f955c075a00 tx=0x7f9550005940 comp rx=0 tx=0).stop 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f955c101d20 msgr2=0x7f955c073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:49.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f956149f700 1 --2- 172.21.15.100:0/3258791066 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f955c101d20 0x7f955c073f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f954c00d950 tx=0x7f954c00dd10 comp rx=0 tx=0).stop 2024-02-14T17:45:49.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f955b7fe700 1 -- 172.21.15.100:0/3258791066 reap_dead start 2024-02-14T17:45:49.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 shutdown_connections 2024-02-14T17:45:49.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 >> 172.21.15.100:0/3258791066 conn(0x7f955c078d40 msgr2=0x7f955c100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:49.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 shutdown_connections 2024-02-14T17:45:49.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:49.339+0000 7f956149f700 1 -- 172.21.15.100:0/3258791066 wait complete. 2024-02-14T17:45:49.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:49 smithi100 ceph-mon[30138]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-14T17:45:49.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:49 smithi100 ceph-mon[39557]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-14T17:45:49.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:49 smithi179 ceph-mon[33633]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-14T17:45:50.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:45:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:45:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:45:51.038 INFO:teuthology.orchestra.run.smithi100.stdout:55834574893 2024-02-14T17:45:51.038 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-14T17:45:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:51 smithi100 ceph-mon[39557]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-14T17:45:51.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:51 smithi100 ceph-mon[30138]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-14T17:45:51.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:51 smithi179 ceph-mon[33633]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-14T17:45:51.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:45:51 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:45:52.251 INFO:teuthology.orchestra.run.smithi100.stdout:163208757272 2024-02-14T17:45:52.252 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-14T17:45:52.586 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:52 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:52.586 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:52 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:52.611 INFO:teuthology.orchestra.run.smithi100.stdout:77309411368 2024-02-14T17:45:52.612 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-14T17:45:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.968+0000 7fd617ec5700 1 -- 172.21.15.100:0/4218939327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6101017b0 msgr2=0x7fd6101057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.968+0000 7fd617ec5700 1 --2- 172.21.15.100:0/4218939327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6101017b0 0x7fd6101057b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd600009990 tx=0x7fd600009ca0 comp rx=0 tx=0).stop 2024-02-14T17:45:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.968+0000 7fd617ec5700 1 -- 172.21.15.100:0/4218939327 shutdown_connections 2024-02-14T17:45:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.968+0000 7fd617ec5700 1 --2- 172.21.15.100:0/4218939327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd610105e90 0x7fd61010da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.968+0000 7fd617ec5700 1 --2- 172.21.15.100:0/4218939327 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd610100e00 0x7fd6101011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:52.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.968+0000 7fd617ec5700 1 --2- 172.21.15.100:0/4218939327 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd6101017b0 0x7fd6101057b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.968+0000 7fd617ec5700 1 -- 172.21.15.100:0/4218939327 >> 172.21.15.100:0/4218939327 conn(0x7fd610076b90 msgr2=0x7fd610076fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.968+0000 7fd617ec5700 1 -- 172.21.15.100:0/4218939327 shutdown_connections 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.968+0000 7fd617ec5700 1 -- 172.21.15.100:0/4218939327 wait complete. 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd617ec5700 1 Processor -- start 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd617ec5700 1 -- start start 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd617ec5700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd610100e00 0x7fd610196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd617ec5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6101017b0 0x7fd610196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd617ec5700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd610105e90 0x7fd61019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd617ec5700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019b560 con 0x7fd610105e90 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd617ec5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019b6d0 con 0x7fd6101017b0 2024-02-14T17:45:52.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd617ec5700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019b890 con 0x7fd610100e00 2024-02-14T17:45:52.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd615c61700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd610100e00 0x7fd610196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:52.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd615c61700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd610100e00 0x7fd610196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:59362/0 (socket says 172.21.15.100:59362) 2024-02-14T17:45:52.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd615c61700 1 -- 172.21.15.100:0/879279653 learned_addr learned my addr 172.21.15.100:0/879279653 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:52.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd616462700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd610105e90 0x7fd61019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:52.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.969+0000 7fd615460700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6101017b0 0x7fd610196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:52.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd615c61700 1 -- 172.21.15.100:0/879279653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6101017b0 msgr2=0x7fd610196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd615c61700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6101017b0 0x7fd610196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd615c61700 1 -- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd610105e90 msgr2=0x7fd61019af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd615c61700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd610105e90 0x7fd61019af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd615c61700 1 -- 172.21.15.100:0/879279653 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd608009c40 con 0x7fd610100e00 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd615460700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6101017b0 0x7fd610196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd615c61700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd610100e00 0x7fd610196680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd60800e5a0 tx=0x7fd60800e8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd606ffd700 1 -- 172.21.15.100:0/879279653 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608003c00 con 0x7fd610100e00 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd606ffd700 1 -- 172.21.15.100:0/879279653 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd6080093a0 con 0x7fd610100e00 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6000095f0 con 0x7fd610100e00 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd606ffd700 1 -- 172.21.15.100:0/879279653 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608007760 con 0x7fd610100e00 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.970+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61006a370 con 0x7fd610100e00 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.971+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd610110350 con 0x7fd610100e00 2024-02-14T17:45:52.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.971+0000 7fd606ffd700 1 -- 172.21.15.100:0/879279653 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7fd608004000 con 0x7fd610100e00 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.972+0000 7fd606ffd700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fd5fc06dac0 0x7fd5fc06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.972+0000 7fd606ffd700 1 -- 172.21.15.100:0/879279653 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fd60808fe70 con 0x7fd610100e00 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.972+0000 7fd615460700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fd5fc06dac0 0x7fd5fc06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.972+0000 7fd606ffd700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967] conn(0x7fd5fc077950 0x7fd5fc079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.972+0000 7fd606ffd700 1 -- 172.21.15.100:0/879279653 --> [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5fc07a4c0 con 0x7fd5fc077950 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.972+0000 7fd606ffd700 1 -- 172.21.15.100:0/879279653 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd608092530 con 0x7fd610100e00 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.972+0000 7fd616462700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967] conn(0x7fd5fc077950 0x7fd5fc079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.972+0000 7fd615460700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fd5fc06dac0 0x7fd5fc06ff80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd600006010 tx=0x7fd600005960 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.972+0000 7fd616462700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967] conn(0x7fd5fc077950 0x7fd5fc079df0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.973+0000 7fd606ffd700 1 -- 172.21.15.100:0/879279653 <== osd.3 v2:172.21.15.100:6826/2018852967 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd5fc07a4c0 con 0x7fd5fc077950 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.989+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 --> [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd6100ff370 con 0x7fd5fc077950 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd606ffd700 1 -- 172.21.15.100:0/879279653 <== osd.3 v2:172.21.15.100:6826/2018852967 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd6100ff370 con 0x7fd5fc077950 2024-02-14T17:45:52.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967] conn(0x7fd5fc077950 msgr2=0x7fd5fc079df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:52.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd617ec5700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:6826/2018852967,v1:172.21.15.100:6827/2018852967] conn(0x7fd5fc077950 0x7fd5fc079df0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:52.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fd5fc06dac0 msgr2=0x7fd5fc06ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:52.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd617ec5700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fd5fc06dac0 0x7fd5fc06ff80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd600006010 tx=0x7fd600005960 comp rx=0 tx=0).stop 2024-02-14T17:45:52.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd610100e00 msgr2=0x7fd610196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:52.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd617ec5700 1 --2- 172.21.15.100:0/879279653 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fd610100e00 0x7fd610196680 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd60800e5a0 tx=0x7fd60800e8b0 comp rx=0 tx=0).stop 2024-02-14T17:45:52.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd616462700 1 -- 172.21.15.100:0/879279653 reap_dead start 2024-02-14T17:45:52.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 shutdown_connections 2024-02-14T17:45:52.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 >> 172.21.15.100:0/879279653 conn(0x7fd610076b90 msgr2=0x7fd6100fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:52.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 shutdown_connections 2024-02-14T17:45:52.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:51.990+0000 7fd617ec5700 1 -- 172.21.15.100:0/879279653 wait complete. 2024-02-14T17:45:52.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:52 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:45:53.601 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:53 smithi100 ceph-mon[39557]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-14T17:45:53.602 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:53 smithi100 ceph-mon[30138]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-14T17:45:53.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:53 smithi179 ceph-mon[33633]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-14T17:45:54.163 INFO:teuthology.orchestra.run.smithi100.stdout:107374182435 2024-02-14T17:45:54.163 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-14T17:45:54.169 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:54.170 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:55.360 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:55.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.361+0000 7f423e640700 1 -- 172.21.15.100:0/288511128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238100f40 msgr2=0x7f4238104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:55.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.361+0000 7f423e640700 1 --2- 172.21.15.100:0/288511128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238100f40 0x7f4238104f90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f422c009990 tx=0x7f422c009ca0 comp rx=0 tx=0).stop 2024-02-14T17:45:55.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.361+0000 7f423e640700 1 -- 172.21.15.100:0/288511128 shutdown_connections 2024-02-14T17:45:55.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.361+0000 7f423e640700 1 --2- 172.21.15.100:0/288511128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4238105670 0x7f423810d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:55.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.361+0000 7f423e640700 1 --2- 172.21.15.100:0/288511128 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4238100590 0x7f4238100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:55.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.361+0000 7f423e640700 1 --2- 172.21.15.100:0/288511128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238100f40 0x7f4238104f90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:55.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.361+0000 7f423e640700 1 -- 172.21.15.100:0/288511128 >> 172.21.15.100:0/288511128 conn(0x7f4238077170 msgr2=0x7f42380fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:55.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.362+0000 7f423e640700 1 -- 172.21.15.100:0/288511128 shutdown_connections 2024-02-14T17:45:55.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.362+0000 7f423e640700 1 -- 172.21.15.100:0/288511128 wait complete. 2024-02-14T17:45:55.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.363+0000 7f423e640700 1 Processor -- start 2024-02-14T17:45:55.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.363+0000 7f423e640700 1 -- start start 2024-02-14T17:45:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.364+0000 7f423e640700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238100590 0x7f42381955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.364+0000 7f423e640700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4238100f40 0x7f4238195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.364+0000 7f423e640700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4238105670 0x7f4238199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.364+0000 7f423e640700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423819a4a0 con 0x7f4238100590 2024-02-14T17:45:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.364+0000 7f423e640700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423819a610 con 0x7f4238100f40 2024-02-14T17:45:55.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.364+0000 7f423e640700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423819a7d0 con 0x7f4238105670 2024-02-14T17:45:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.364+0000 7f4237fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238100590 0x7f42381955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.364+0000 7f4237fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238100590 0x7f42381955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38526/0 (socket says 172.21.15.100:38526) 2024-02-14T17:45:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.364+0000 7f4237fff700 1 -- 172.21.15.100:0/2209426901 learned_addr learned my addr 172.21.15.100:0/2209426901 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.364+0000 7f423cbdd700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4238105670 0x7f4238199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.365+0000 7f42377fe700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4238100f40 0x7f4238195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.365+0000 7f423cbdd700 1 -- 172.21.15.100:0/2209426901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4238100f40 msgr2=0x7f4238195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.365+0000 7f423cbdd700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4238100f40 0x7f4238195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:55.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.365+0000 7f423cbdd700 1 -- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238100590 msgr2=0x7f42381955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:55.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.365+0000 7f423cbdd700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4238100590 0x7f42381955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:55.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.365+0000 7f423cbdd700 1 -- 172.21.15.100:0/2209426901 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0095f0 con 0x7f4238105670 2024-02-14T17:45:55.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.366+0000 7f423cbdd700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4238105670 0x7f4238199e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f423000efc0 tx=0x7f423000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:55.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.366+0000 7f42357fa700 1 -- 172.21.15.100:0/2209426901 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4230018070 con 0x7f4238105670 2024-02-14T17:45:55.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.367+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423819aa50 con 0x7f4238105670 2024-02-14T17:45:55.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.367+0000 7f42357fa700 1 -- 172.21.15.100:0/2209426901 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f423000f040 con 0x7f4238105670 2024-02-14T17:45:55.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.367+0000 7f42357fa700 1 -- 172.21.15.100:0/2209426901 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423000cc50 con 0x7f4238105670 2024-02-14T17:45:55.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.367+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42381a2140 con 0x7f4238105670 2024-02-14T17:45:55.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.368+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f423810f290 con 0x7f4238105670 2024-02-14T17:45:55.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.373+0000 7f42357fa700 1 -- 172.21.15.100:0/2209426901 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f4230003a00 con 0x7f4238105670 2024-02-14T17:45:55.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.374+0000 7f42357fa700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f4220071ed0 0x7f4220074390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:55.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.374+0000 7f42357fa700 1 -- 172.21.15.100:0/2209426901 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f423008e1b0 con 0x7f4238105670 2024-02-14T17:45:55.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.374+0000 7f42357fa700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533] conn(0x7f422007be60 0x7f422007e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:55.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.374+0000 7f42357fa700 1 -- 172.21.15.100:0/2209426901 --> [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f422007e9d0 con 0x7f422007be60 2024-02-14T17:45:55.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.374+0000 7f42357fa700 1 -- 172.21.15.100:0/2209426901 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f423008e560 con 0x7f4238105670 2024-02-14T17:45:55.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.375+0000 7f4237fff700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f4220071ed0 0x7f4220074390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:55.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.375+0000 7f42377fe700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533] conn(0x7f422007be60 0x7f422007e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:55.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.375+0000 7f4237fff700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f4220071ed0 0x7f4220074390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f422800ad90 tx=0x7f4228010040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:55.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.376+0000 7f42377fe700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533] conn(0x7f422007be60 0x7f422007e300 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:55.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.377+0000 7f42357fa700 1 -- 172.21.15.100:0/2209426901 <== osd.4 v2:172.21.15.179:6800/952874533 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f422007e9d0 con 0x7f422007be60 2024-02-14T17:45:55.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.398+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 --> [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4238103bb0 con 0x7f422007be60 2024-02-14T17:45:55.400 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.398+0000 7f42357fa700 1 -- 172.21.15.100:0/2209426901 <== osd.4 v2:172.21.15.179:6800/952874533 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4238103bb0 con 0x7f422007be60 2024-02-14T17:45:55.401 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.398+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 >> [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533] conn(0x7f422007be60 msgr2=0x7f422007e300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:55.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.398+0000 7f423e640700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.179:6800/952874533,v1:172.21.15.179:6801/952874533] conn(0x7f422007be60 0x7f422007e300 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:55.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.399+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f4220071ed0 msgr2=0x7f4220074390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:55.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.399+0000 7f423e640700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f4220071ed0 0x7f4220074390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f422800ad90 tx=0x7f4228010040 comp rx=0 tx=0).stop 2024-02-14T17:45:55.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.399+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4238105670 msgr2=0x7f4238199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:55.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.399+0000 7f423e640700 1 --2- 172.21.15.100:0/2209426901 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f4238105670 0x7f4238199e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f423000efc0 tx=0x7f423000c3a0 comp rx=0 tx=0).stop 2024-02-14T17:45:55.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.399+0000 7f423cbdd700 1 -- 172.21.15.100:0/2209426901 reap_dead start 2024-02-14T17:45:55.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.399+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 shutdown_connections 2024-02-14T17:45:55.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.399+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 >> 172.21.15.100:0/2209426901 conn(0x7f4238077170 msgr2=0x7f4238103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:55.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.399+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 shutdown_connections 2024-02-14T17:45:55.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:55.399+0000 7f423e640700 1 -- 172.21.15.100:0/2209426901 wait complete. 2024-02-14T17:45:55.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:55 smithi100 ceph-mon[30138]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:45:55.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:55 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:55.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:55 smithi100 ceph-mon[39557]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:45:55.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:55 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:55.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:55 smithi179 ceph-mon[33633]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:45:55.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:55 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:45:57.601 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:57 smithi100 ceph-mon[30138]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:45:57.601 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:57 smithi100 ceph-mon[39557]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:45:57.634 INFO:teuthology.orchestra.run.smithi100.stdout:128849018909 2024-02-14T17:45:57.634 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-14T17:45:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:57 smithi179 ceph-mon[33633]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:45:58.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.144+0000 7fbbdd472700 1 -- 172.21.15.100:0/2712550172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8101530 msgr2=0x7fbbd810d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.144+0000 7fbbdd472700 1 --2- 172.21.15.100:0/2712550172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8101530 0x7fbbd810d9c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000b0a0 tx=0x7fbbd000b3b0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.144+0000 7fbbdd472700 1 -- 172.21.15.100:0/2712550172 shutdown_connections 2024-02-14T17:45:58.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.144+0000 7fbbdd472700 1 --2- 172.21.15.100:0/2712550172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd8106b80 0x7fbbd8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.144+0000 7fbbdd472700 1 --2- 172.21.15.100:0/2712550172 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8101530 0x7fbbd810d9c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.144+0000 7fbbdd472700 1 --2- 172.21.15.100:0/2712550172 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd8100b70 0x7fbbd8100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.144+0000 7fbbdd472700 1 -- 172.21.15.100:0/2712550172 >> 172.21.15.100:0/2712550172 conn(0x7fbbd8077aa0 msgr2=0x7fbbd8077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.144+0000 7fbbdd472700 1 -- 172.21.15.100:0/2712550172 shutdown_connections 2024-02-14T17:45:58.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.144+0000 7fbbdd472700 1 -- 172.21.15.100:0/2712550172 wait complete. 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.144+0000 7fbbdd472700 1 Processor -- start 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbdd472700 1 -- start start 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbdd472700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8100b70 0x7fbbd81027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbdd472700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd8101530 0x7fbbd8102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbdd472700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd8106b80 0x7fbbd8103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbdd472700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd81a5b00 con 0x7fbbd8100b70 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbdd472700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd81a5c70 con 0x7fbbd8106b80 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbdd472700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd81a5e30 con 0x7fbbd8101530 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd67fc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd8101530 0x7fbbd8102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8100b70 0x7fbbd81027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd67fc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd8101530 0x7fbbd8102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:41356/0 (socket says 172.21.15.100:41356) 2024-02-14T17:45:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd67fc700 1 -- 172.21.15.100:0/2611783699 learned_addr learned my addr 172.21.15.100:0/2611783699 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd6ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8100b70 0x7fbbd81027b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38538/0 (socket says 172.21.15.100:38538) 2024-02-14T17:45:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd77fe700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd8106b80 0x7fbbd8103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd67fc700 1 -- 172.21.15.100:0/2611783699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd8106b80 msgr2=0x7fbbd8103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd67fc700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbd8106b80 0x7fbbd8103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd67fc700 1 -- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8100b70 msgr2=0x7fbbd81027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd67fc700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8100b70 0x7fbbd81027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd67fc700 1 -- 172.21.15.100:0/2611783699 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd0009d00 con 0x7fbbd8101530 2024-02-14T17:45:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd6ffd700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbd8100b70 0x7fbbd81027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:45:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.145+0000 7fbbd67fc700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd8101530 0x7fbbd8102cf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00e9a0 tx=0x7fbbcc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.146+0000 7fbbbffff700 1 -- 172.21.15.100:0/2611783699 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc009790 con 0x7fbbd8101530 2024-02-14T17:45:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.146+0000 7fbbbffff700 1 -- 172.21.15.100:0/2611783699 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fbbcc00ce90 con 0x7fbbd8101530 2024-02-14T17:45:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.146+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd81a6110 con 0x7fbbd8101530 2024-02-14T17:45:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.146+0000 7fbbbffff700 1 -- 172.21.15.100:0/2611783699 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc018420 con 0x7fbbd8101530 2024-02-14T17:45:58.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.146+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd81a6610 con 0x7fbbd8101530 2024-02-14T17:45:58.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.146+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbd8110300 con 0x7fbbd8101530 2024-02-14T17:45:58.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.147+0000 7fbbbffff700 1 -- 172.21.15.100:0/2611783699 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7fbbcc00ca10 con 0x7fbbd8101530 2024-02-14T17:45:58.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.147+0000 7fbbbffff700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fbbc006dac0 0x7fbbc006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:58.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.148+0000 7fbbbffff700 1 -- 172.21.15.100:0/2611783699 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fbbcc014070 con 0x7fbbd8101530 2024-02-14T17:45:58.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.148+0000 7fbbd6ffd700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fbbc006dac0 0x7fbbc006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:58.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.148+0000 7fbbbffff700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257] conn(0x7fbbc0077950 0x7fbbc0079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:58.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.148+0000 7fbbbffff700 1 -- 172.21.15.100:0/2611783699 --> [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbc007a4c0 con 0x7fbbc0077950 2024-02-14T17:45:58.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.148+0000 7fbbbffff700 1 -- 172.21.15.100:0/2611783699 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbcc0922c0 con 0x7fbbd8101530 2024-02-14T17:45:58.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.148+0000 7fbbd6ffd700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fbbc006dac0 0x7fbbc006ff80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800afa0 tx=0x7fbbc800a380 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:58.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.148+0000 7fbbd77fe700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257] conn(0x7fbbc0077950 0x7fbbc0079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:58.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.148+0000 7fbbd77fe700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257] conn(0x7fbbc0077950 0x7fbbc0079df0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:58.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.149+0000 7fbbbffff700 1 -- 172.21.15.100:0/2611783699 <== osd.6 v2:172.21.15.179:6816/2309639257 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbbc007a4c0 con 0x7fbbc0077950 2024-02-14T17:45:58.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.165+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 --> [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbbd806a280 con 0x7fbbc0077950 2024-02-14T17:45:58.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbbffff700 1 -- 172.21.15.100:0/2611783699 <== osd.6 v2:172.21.15.179:6816/2309639257 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbbd806a280 con 0x7fbbc0077950 2024-02-14T17:45:58.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 >> [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257] conn(0x7fbbc0077950 msgr2=0x7fbbc0079df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbdd472700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.179:6816/2309639257,v1:172.21.15.179:6817/2309639257] conn(0x7fbbc0077950 0x7fbbc0079df0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fbbc006dac0 msgr2=0x7fbbc006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbdd472700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fbbc006dac0 0x7fbbc006ff80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800afa0 tx=0x7fbbc800a380 comp rx=0 tx=0).stop 2024-02-14T17:45:58.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd8101530 msgr2=0x7fbbd8102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbdd472700 1 --2- 172.21.15.100:0/2611783699 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fbbd8101530 0x7fbbd8102cf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00e9a0 tx=0x7fbbcc00ecb0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbd77fe700 1 -- 172.21.15.100:0/2611783699 reap_dead start 2024-02-14T17:45:58.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 shutdown_connections 2024-02-14T17:45:58.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 >> 172.21.15.100:0/2611783699 conn(0x7fbbd8077aa0 msgr2=0x7fbbd8069920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:58.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 shutdown_connections 2024-02-14T17:45:58.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.166+0000 7fbbdd472700 1 -- 172.21.15.100:0/2611783699 wait complete. 2024-02-14T17:45:58.399 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:58.399 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:58.635 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:45:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.638+0000 7f5854959700 1 -- 172.21.15.100:0/4127507374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5850101cf0 msgr2=0x7f5850102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.638+0000 7f5854959700 1 --2- 172.21.15.100:0/4127507374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5850101cf0 0x7f5850102170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f58400098f0 tx=0x7f5840009c00 comp rx=0 tx=0).stop 2024-02-14T17:45:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.638+0000 7f5854959700 1 -- 172.21.15.100:0/4127507374 shutdown_connections 2024-02-14T17:45:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.638+0000 7f5854959700 1 --2- 172.21.15.100:0/4127507374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5850107dd0 0x7f58501081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.638+0000 7f5854959700 1 --2- 172.21.15.100:0/4127507374 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58501026b0 0x7f585010eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.638+0000 7f5854959700 1 --2- 172.21.15.100:0/4127507374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5850101cf0 0x7f5850102170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.638+0000 7f5854959700 1 -- 172.21.15.100:0/4127507374 >> 172.21.15.100:0/4127507374 conn(0x7f5850078d30 msgr2=0x7f5850079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.638+0000 7f5854959700 1 -- 172.21.15.100:0/4127507374 shutdown_connections 2024-02-14T17:45:58.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.638+0000 7f5854959700 1 -- 172.21.15.100:0/4127507374 wait complete. 2024-02-14T17:45:58.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.639+0000 7f5854959700 1 Processor -- start 2024-02-14T17:45:58.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.639+0000 7f5854959700 1 -- start start 2024-02-14T17:45:58.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.639+0000 7f5854959700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5850101cf0 0x7f585019d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:58.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.639+0000 7f5854959700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58501026b0 0x7f585019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:58.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.639+0000 7f5854959700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5850107dd0 0x7f5850197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:58.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.639+0000 7f5854959700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5850197bf0 con 0x7f5850101cf0 2024-02-14T17:45:58.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.639+0000 7f5854959700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5850197d30 con 0x7f5850107dd0 2024-02-14T17:45:58.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.639+0000 7f5854959700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5850197ef0 con 0x7f58501026b0 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.639+0000 7f584e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5850101cf0 0x7f585019d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f584e59c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5850101cf0 0x7f585019d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38542/0 (socket says 172.21.15.100:38542) 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f584e59c700 1 -- 172.21.15.100:0/3545775301 learned_addr learned my addr 172.21.15.100:0/3545775301 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.639+0000 7f584dd9b700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58501026b0 0x7f585019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f584ed9d700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5850107dd0 0x7f5850197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f584e59c700 1 -- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58501026b0 msgr2=0x7f585019d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f584e59c700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58501026b0 0x7f585019d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f584e59c700 1 -- 172.21.15.100:0/3545775301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5850107dd0 msgr2=0x7f5850197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f584e59c700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5850107dd0 0x7f5850197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f584e59c700 1 -- 172.21.15.100:0/3545775301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5848009d00 con 0x7f5850101cf0 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f584dd9b700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58501026b0 0x7f585019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f584e59c700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5850101cf0 0x7f585019d3d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f584400c920 tx=0x7f584400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f583f7fe700 1 -- 172.21.15.100:0/3545775301 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844004a30 con 0x7f5850101cf0 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f583f7fe700 1 -- 172.21.15.100:0/3545775301 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f58440054a0 con 0x7f5850101cf0 2024-02-14T17:45:58.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f5854959700 1 -- 172.21.15.100:0/3545775301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5840009520 con 0x7f5850101cf0 2024-02-14T17:45:58.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f583f7fe700 1 -- 172.21.15.100:0/3545775301 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844007ab0 con 0x7f5850101cf0 2024-02-14T17:45:58.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.640+0000 7f5854959700 1 -- 172.21.15.100:0/3545775301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58501a42a0 con 0x7f5850101cf0 2024-02-14T17:45:58.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.642+0000 7f5854959700 1 -- 172.21.15.100:0/3545775301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585010a260 con 0x7f5850101cf0 2024-02-14T17:45:58.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.642+0000 7f583f7fe700 1 -- 172.21.15.100:0/3545775301 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f5844007c10 con 0x7f5850101cf0 2024-02-14T17:45:58.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.642+0000 7f583f7fe700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f583806dac0 0x7f583806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:45:58.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.642+0000 7f583f7fe700 1 -- 172.21.15.100:0/3545775301 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f584408d780 con 0x7f5850101cf0 2024-02-14T17:45:58.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.643+0000 7f584dd9b700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f583806dac0 0x7f583806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:45:58.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.643+0000 7f584dd9b700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f583806dac0 0x7f583806ff80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5840009fd0 tx=0x7f5840019040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:45:58.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.648+0000 7f583f7fe700 1 -- 172.21.15.100:0/3545775301 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5844056b30 con 0x7f5850101cf0 2024-02-14T17:45:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.865+0000 7f5854959700 1 -- 172.21.15.100:0/3545775301 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5850068930 con 0x7f5850101cf0 2024-02-14T17:45:58.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.865+0000 7f583f7fe700 1 -- 172.21.15.100:0/3545775301 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f584405a1f0 con 0x7f5850101cf0 2024-02-14T17:45:58.867 INFO:teuthology.orchestra.run.smithi100.stdout:34359738420 2024-02-14T17:45:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.867+0000 7f5854959700 1 -- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f583806dac0 msgr2=0x7f583806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.867+0000 7f5854959700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f583806dac0 0x7f583806ff80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5840009fd0 tx=0x7f5840019040 comp rx=0 tx=0).stop 2024-02-14T17:45:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.867+0000 7f5854959700 1 -- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5850101cf0 msgr2=0x7f585019d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:45:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.867+0000 7f5854959700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5850101cf0 0x7f585019d3d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f584400c920 tx=0x7f584400cc30 comp rx=0 tx=0).stop 2024-02-14T17:45:58.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.867+0000 7f5854959700 1 -- 172.21.15.100:0/3545775301 shutdown_connections 2024-02-14T17:45:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.867+0000 7f5854959700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5850107dd0 0x7f5850197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.867+0000 7f5854959700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f583806dac0 0x7f583806ff80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.867+0000 7f5854959700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5850101cf0 0x7f585019d3d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.867+0000 7f5854959700 1 --2- 172.21.15.100:0/3545775301 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f58501026b0 0x7f585019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:45:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.868+0000 7f5854959700 1 -- 172.21.15.100:0/3545775301 >> 172.21.15.100:0/3545775301 conn(0x7f5850078d30 msgr2=0x7f5850100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:45:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.868+0000 7f5854959700 1 -- 172.21.15.100:0/3545775301 shutdown_connections 2024-02-14T17:45:58.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:45:58.868+0000 7f5854959700 1 -- 172.21.15.100:0/3545775301 wait complete. 2024-02-14T17:45:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:59 smithi100 ceph-mon[30138]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:45:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:45:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3545775301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T17:45:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:59 smithi100 ceph-mon[39557]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:45:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:45:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3545775301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T17:45:59.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:59 smithi179 ceph-mon[33633]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T17:45:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:45:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3545775301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-14T17:46:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.444+0000 7f048d48c700 1 -- 172.21.15.100:0/2907377459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0488100f60 msgr2=0x7f0488112500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.444+0000 7f048d48c700 1 --2- 172.21.15.100:0/2907377459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0488100f60 0x7f0488112500 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f047c0098f0 tx=0x7f047c009c00 comp rx=0 tx=0).stop 2024-02-14T17:46:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.444+0000 7f048d48c700 1 -- 172.21.15.100:0/2907377459 shutdown_connections 2024-02-14T17:46:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.444+0000 7f048d48c700 1 --2- 172.21.15.100:0/2907377459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04881005b0 0x7f0488100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.444+0000 7f048d48c700 1 --2- 172.21.15.100:0/2907377459 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f0488112a40 0x7f0488114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:00.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.444+0000 7f048d48c700 1 --2- 172.21.15.100:0/2907377459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0488100f60 0x7f0488112500 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.444+0000 7f048d48c700 1 -- 172.21.15.100:0/2907377459 >> 172.21.15.100:0/2907377459 conn(0x7f0488075eb0 msgr2=0x7f04880762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.445+0000 7f048d48c700 1 -- 172.21.15.100:0/2907377459 shutdown_connections 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.445+0000 7f048d48c700 1 -- 172.21.15.100:0/2907377459 wait complete. 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.445+0000 7f048d48c700 1 Processor -- start 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.445+0000 7f048d48c700 1 -- start start 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f048d48c700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881005b0 0x7f048819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f048d48c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0488100f60 0x7f048819e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f048d48c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0488112a40 0x7f04881a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f048d48c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04881a2d40 con 0x7f0488112a40 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f048d48c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04881a2eb0 con 0x7f0488100f60 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f048d48c700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04881a3070 con 0x7f04881005b0 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f0486ffd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881005b0 0x7f048819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:00.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04877fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0488112a40 0x7f04881a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04877fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0488112a40 0x7f04881a2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38560/0 (socket says 172.21.15.100:38560) 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f0486ffd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881005b0 0x7f048819de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:41390/0 (socket says 172.21.15.100:41390) 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04877fe700 1 -- 172.21.15.100:0/1300149813 learned_addr learned my addr 172.21.15.100:0/1300149813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04867fc700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0488100f60 0x7f048819e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04877fe700 1 -- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881005b0 msgr2=0x7f048819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04877fe700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881005b0 0x7f048819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04877fe700 1 -- 172.21.15.100:0/1300149813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0488100f60 msgr2=0x7f048819e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04877fe700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0488100f60 0x7f048819e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04877fe700 1 -- 172.21.15.100:0/1300149813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480009d00 con 0x7f0488112a40 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f0486ffd700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881005b0 0x7f048819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04867fc700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0488100f60 0x7f048819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f04877fe700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0488112a40 0x7f04881a2730 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f048000c3d0 tx=0x7f048000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f046ffff700 1 -- 172.21.15.100:0/1300149813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048000e070 con 0x7f0488112a40 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f048d48c700 1 -- 172.21.15.100:0/1300149813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c009520 con 0x7f0488112a40 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f046ffff700 1 -- 172.21.15.100:0/1300149813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f048000f040 con 0x7f0488112a40 2024-02-14T17:46:00.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f046ffff700 1 -- 172.21.15.100:0/1300149813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480004460 con 0x7f0488112a40 2024-02-14T17:46:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.446+0000 7f048d48c700 1 -- 172.21.15.100:0/1300149813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04881aa9e0 con 0x7f0488112a40 2024-02-14T17:46:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.447+0000 7f048d48c700 1 -- 172.21.15.100:0/1300149813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0488068930 con 0x7f0488112a40 2024-02-14T17:46:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.448+0000 7f046ffff700 1 -- 172.21.15.100:0/1300149813 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f04800045c0 con 0x7f0488112a40 2024-02-14T17:46:00.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.448+0000 7f046ffff700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f047006dac0 0x7f047006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:00.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.448+0000 7f0486ffd700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f047006dac0 0x7f047006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:00.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.448+0000 7f046ffff700 1 -- 172.21.15.100:0/1300149813 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f048008d4d0 con 0x7f0488112a40 2024-02-14T17:46:00.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.449+0000 7f0486ffd700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f047006dac0 0x7f047006ff80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0478009de0 tx=0x7f0478009190 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:00.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.453+0000 7f046ffff700 1 -- 172.21.15.100:0/1300149813 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04800568e0 con 0x7f0488112a40 2024-02-14T17:46:00.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:46:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:00.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.650+0000 7f048d48c700 1 -- 172.21.15.100:0/1300149813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0488117860 con 0x7f0488112a40 2024-02-14T17:46:00.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.651+0000 7f046ffff700 1 -- 172.21.15.100:0/1300149813 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0480059fa0 con 0x7f0488112a40 2024-02-14T17:46:00.652 INFO:teuthology.orchestra.run.smithi100.stdout:231928234000 2024-02-14T17:46:00.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 -- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f047006dac0 msgr2=0x7f047006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:00.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f047006dac0 0x7f047006ff80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0478009de0 tx=0x7f0478009190 comp rx=0 tx=0).stop 2024-02-14T17:46:00.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 -- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0488112a40 msgr2=0x7f04881a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:00.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0488112a40 0x7f04881a2730 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f048000c3d0 tx=0x7f048000c6e0 comp rx=0 tx=0).stop 2024-02-14T17:46:00.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 -- 172.21.15.100:0/1300149813 shutdown_connections 2024-02-14T17:46:00.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f047006dac0 0x7f047006ff80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:00.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0488112a40 0x7f04881a2730 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:00.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f04881005b0 0x7f048819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:00.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 --2- 172.21.15.100:0/1300149813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0488100f60 0x7f048819e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:00.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 -- 172.21.15.100:0/1300149813 >> 172.21.15.100:0/1300149813 conn(0x7f0488075eb0 msgr2=0x7f04881104f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:00.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 -- 172.21.15.100:0/1300149813 shutdown_connections 2024-02-14T17:46:00.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:00.653+0000 7f048d48c700 1 -- 172.21.15.100:0/1300149813 wait complete. 2024-02-14T17:46:01.642 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:01 smithi100 ceph-mon[39557]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-14T17:46:01.642 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1300149813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-14T17:46:01.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:01 smithi100 ceph-mon[30138]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-14T17:46:01.643 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1300149813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-14T17:46:01.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:46:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:46:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:01 smithi179 ceph-mon[33633]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-14T17:46:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1300149813' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-14T17:46:02.157 INFO:teuthology.orchestra.run.smithi100.stdout:197568495638 2024-02-14T17:46:02.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-14T17:46:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:02 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:02 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:02 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:03.635 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:03 smithi100 ceph-mon[39557]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:46:03.636 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:03 smithi100 ceph-mon[30138]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:46:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:03 smithi179 ceph-mon[33633]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T17:46:05.299 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928233998 got 231928234000 for osd.7 2024-02-14T17:46:05.299 DEBUG:teuthology.parallel:result is None 2024-02-14T17:46:05.531 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:05 smithi100 ceph-mon[39557]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:05.531 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:05 smithi100 ceph-mon[30138]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:05 smithi179 ceph-mon[33633]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 -- 172.21.15.100:0/3860347533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf18106b80 msgr2=0x7fcf18106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 --2- 172.21.15.100:0/3860347533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf18106b80 0x7fcf18106f60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009990 tx=0x7fcf14009ca0 comp rx=0 tx=0).stop 2024-02-14T17:46:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 -- 172.21.15.100:0/3860347533 shutdown_connections 2024-02-14T17:46:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 --2- 172.21.15.100:0/3860347533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf18101530 0x7fcf1810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 --2- 172.21.15.100:0/3860347533 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf18106b80 0x7fcf18106f60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 --2- 172.21.15.100:0/3860347533 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf18100b70 0x7fcf18100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:05.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 -- 172.21.15.100:0/3860347533 >> 172.21.15.100:0/3860347533 conn(0x7fcf18077aa0 msgr2=0x7fcf18077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 -- 172.21.15.100:0/3860347533 shutdown_connections 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 -- 172.21.15.100:0/3860347533 wait complete. 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 Processor -- start 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.994+0000 7fcf1f786700 1 -- start start 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1f786700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf18100b70 0x7fcf18196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1f786700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf18101530 0x7fcf18196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1f786700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf18106b80 0x7fcf1819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1f786700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1819b450 con 0x7fcf18101530 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1f786700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1819b5c0 con 0x7fcf18100b70 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1f786700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1819b780 con 0x7fcf18106b80 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1cd21700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf18101530 0x7fcf18196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1cd21700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf18101530 0x7fcf18196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38186/0 (socket says 172.21.15.100:38186) 2024-02-14T17:46:05.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1cd21700 1 -- 172.21.15.100:0/933823346 learned_addr learned my addr 172.21.15.100:0/933823346 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1dd23700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf18106b80 0x7fcf1819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1d522700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf18100b70 0x7fcf18196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1cd21700 1 -- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf18106b80 msgr2=0x7fcf1819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1cd21700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf18106b80 0x7fcf1819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1cd21700 1 -- 172.21.15.100:0/933823346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf18100b70 msgr2=0x7fcf18196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1cd21700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf18100b70 0x7fcf18196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1cd21700 1 -- 172.21.15.100:0/933823346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf08009520 con 0x7fcf18101530 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf1cd21700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf18101530 0x7fcf18196ab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800eb30 tx=0x7fcf0800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf0e7fc700 1 -- 172.21.15.100:0/933823346 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08009880 con 0x7fcf18101530 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf0e7fc700 1 -- 172.21.15.100:0/933823346 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcf080044a0 con 0x7fcf18101530 2024-02-14T17:46:06.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.995+0000 7fcf0e7fc700 1 -- 172.21.15.100:0/933823346 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08003ca0 con 0x7fcf18101530 2024-02-14T17:46:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.996+0000 7fcf1f786700 1 -- 172.21.15.100:0/933823346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf140095f0 con 0x7fcf18101530 2024-02-14T17:46:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.996+0000 7fcf1f786700 1 -- 172.21.15.100:0/933823346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf18069920 con 0x7fcf18101530 2024-02-14T17:46:06.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.996+0000 7fcf1f786700 1 -- 172.21.15.100:0/933823346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf18068930 con 0x7fcf18101530 2024-02-14T17:46:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.997+0000 7fcf0e7fc700 1 -- 172.21.15.100:0/933823346 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7fcf08003e00 con 0x7fcf18101530 2024-02-14T17:46:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.997+0000 7fcf0e7fc700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcf0406dac0 0x7fcf0406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.997+0000 7fcf1d522700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcf0406dac0 0x7fcf0406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:06.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.997+0000 7fcf0e7fc700 1 -- 172.21.15.100:0/933823346 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fcf08014070 con 0x7fcf18101530 2024-02-14T17:46:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:05.998+0000 7fcf1d522700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcf0406dac0 0x7fcf0406ff80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcf14006010 tx=0x7fcf14005960 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:06.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.002+0000 7fcf0e7fc700 1 -- 172.21.15.100:0/933823346 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf08057090 con 0x7fcf18101530 2024-02-14T17:46:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.185+0000 7f753292b700 1 -- 172.21.15.100:0/2068558435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f752c107ca0 msgr2=0x7f752c108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.185+0000 7f753292b700 1 --2- 172.21.15.100:0/2068558435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f752c107ca0 0x7f752c108080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098f0 tx=0x7f751c009c00 comp rx=0 tx=0).stop 2024-02-14T17:46:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.185+0000 7f753292b700 1 -- 172.21.15.100:0/2068558435 shutdown_connections 2024-02-14T17:46:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.185+0000 7f753292b700 1 --2- 172.21.15.100:0/2068558435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f752c06ad80 0x7f752c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.185+0000 7f753292b700 1 --2- 172.21.15.100:0/2068558435 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f752c107ca0 0x7f752c108080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.185+0000 7f753292b700 1 --2- 172.21.15.100:0/2068558435 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752c06a3c0 0x7f752c06a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.185+0000 7f753292b700 1 -- 172.21.15.100:0/2068558435 >> 172.21.15.100:0/2068558435 conn(0x7f752c077b80 msgr2=0x7f752c077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.185+0000 7f753292b700 1 -- 172.21.15.100:0/2068558435 shutdown_connections 2024-02-14T17:46:06.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.185+0000 7f753292b700 1 -- 172.21.15.100:0/2068558435 wait complete. 2024-02-14T17:46:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.186+0000 7f753292b700 1 Processor -- start 2024-02-14T17:46:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.186+0000 7f753292b700 1 -- start start 2024-02-14T17:46:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.186+0000 7f753292b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f752c06a3c0 0x7f752c1976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:06.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.186+0000 7f753292b700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f752c06ad80 0x7f752c197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.186+0000 7f753292b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752c107ca0 0x7f752c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.186+0000 7f753292b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c19c5c0 con 0x7f752c107ca0 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.186+0000 7f753292b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c19c730 con 0x7f752c06a3c0 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.186+0000 7f753292b700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c19c8f0 con 0x7f752c06ad80 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.186+0000 7f752b7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f752c06ad80 0x7f752c197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f7530ec8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752c107ca0 0x7f752c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f7530ec8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752c107ca0 0x7f752c19bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38210/0 (socket says 172.21.15.100:38210) 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f7530ec8700 1 -- 172.21.15.100:0/480599905 learned_addr learned my addr 172.21.15.100:0/480599905 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f752b7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f752c06ad80 0x7f752c197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:56692/0 (socket says 172.21.15.100:56692) 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f752bfff700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f752c06a3c0 0x7f752c1976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f7530ec8700 1 -- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f752c06ad80 msgr2=0x7f752c197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:06.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f7530ec8700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f752c06ad80 0x7f752c197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f7530ec8700 1 -- 172.21.15.100:0/480599905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f752c06a3c0 msgr2=0x7f752c1976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f7530ec8700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f752c06a3c0 0x7f752c1976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f7530ec8700 1 -- 172.21.15.100:0/480599905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75200095f0 con 0x7f752c107ca0 2024-02-14T17:46:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f752bfff700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f752c06a3c0 0x7f752c1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:46:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f752b7fe700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f752c06ad80 0x7f752c197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:46:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f7530ec8700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752c107ca0 0x7f752c19bfb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f752400ef30 tx=0x7f752400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f75297fa700 1 -- 172.21.15.100:0/480599905 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524018070 con 0x7f752c107ca0 2024-02-14T17:46:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.187+0000 7f753292b700 1 -- 172.21.15.100:0/480599905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751c009520 con 0x7f752c107ca0 2024-02-14T17:46:06.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.188+0000 7f75297fa700 1 -- 172.21.15.100:0/480599905 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f752400f040 con 0x7f752c107ca0 2024-02-14T17:46:06.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.188+0000 7f75297fa700 1 -- 172.21.15.100:0/480599905 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400cb60 con 0x7f752c107ca0 2024-02-14T17:46:06.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.188+0000 7f753292b700 1 -- 172.21.15.100:0/480599905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c100b50 con 0x7f752c107ca0 2024-02-14T17:46:06.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.189+0000 7f75297fa700 1 -- 172.21.15.100:0/480599905 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f7524004020 con 0x7f752c107ca0 2024-02-14T17:46:06.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.189+0000 7f75297fa700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f751406dac0 0x7f751406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:06.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.190+0000 7f75297fa700 1 -- 172.21.15.100:0/480599905 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f752408cd20 con 0x7f752c107ca0 2024-02-14T17:46:06.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.190+0000 7f752bfff700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f751406dac0 0x7f751406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:06.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.190+0000 7f752bfff700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f751406dac0 0x7f751406ff80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098c0 tx=0x7f751c019040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:06.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.190+0000 7f753292b700 1 -- 172.21.15.100:0/480599905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f752c068930 con 0x7f752c107ca0 2024-02-14T17:46:06.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.198+0000 7f75297fa700 1 -- 172.21.15.100:0/480599905 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7524056130 con 0x7f752c107ca0 2024-02-14T17:46:06.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.219+0000 7fcf1f786700 1 -- 172.21.15.100:0/933823346 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcf18110070 con 0x7fcf18101530 2024-02-14T17:46:06.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.220+0000 7fcf0e7fc700 1 -- 172.21.15.100:0/933823346 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcf0805fe60 con 0x7fcf18101530 2024-02-14T17:46:06.221 INFO:teuthology.orchestra.run.smithi100.stdout:77309411371 2024-02-14T17:46:06.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 -- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcf0406dac0 msgr2=0x7fcf0406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcf0406dac0 0x7fcf0406ff80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcf14006010 tx=0x7fcf14005960 comp rx=0 tx=0).stop 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 -- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf18101530 msgr2=0x7fcf18196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf18101530 0x7fcf18196ab0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800eb30 tx=0x7fcf0800ee40 comp rx=0 tx=0).stop 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 -- 172.21.15.100:0/933823346 shutdown_connections 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf18100b70 0x7fcf18196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcf0406dac0 0x7fcf0406ff80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcf18106b80 0x7fcf1819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 --2- 172.21.15.100:0/933823346 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcf18101530 0x7fcf18196ab0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 -- 172.21.15.100:0/933823346 >> 172.21.15.100:0/933823346 conn(0x7fcf18077aa0 msgr2=0x7fcf181077b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 -- 172.21.15.100:0/933823346 shutdown_connections 2024-02-14T17:46:06.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.222+0000 7fcf1f786700 1 -- 172.21.15.100:0/933823346 wait complete. 2024-02-14T17:46:06.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.411+0000 7f753292b700 1 -- 172.21.15.100:0/480599905 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f752c1111e0 con 0x7f752c107ca0 2024-02-14T17:46:06.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.411+0000 7f75297fa700 1 -- 172.21.15.100:0/480599905 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75240597f0 con 0x7f752c107ca0 2024-02-14T17:46:06.413 INFO:teuthology.orchestra.run.smithi100.stdout:107374182438 2024-02-14T17:46:06.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 -- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f751406dac0 msgr2=0x7f751406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:06.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f751406dac0 0x7f751406ff80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f751c0098c0 tx=0x7f751c019040 comp rx=0 tx=0).stop 2024-02-14T17:46:06.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 -- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752c107ca0 msgr2=0x7f752c19bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:06.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752c107ca0 0x7f752c19bfb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f752400ef30 tx=0x7f752400c3a0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 -- 172.21.15.100:0/480599905 shutdown_connections 2024-02-14T17:46:06.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f752c06a3c0 0x7f752c1976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f751406dac0 0x7f751406ff80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f752c107ca0 0x7f752c19bfb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 --2- 172.21.15.100:0/480599905 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f752c06ad80 0x7f752c197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:06.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 -- 172.21.15.100:0/480599905 >> 172.21.15.100:0/480599905 conn(0x7f752c077b80 msgr2=0x7f752c105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:06.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 -- 172.21.15.100:0/480599905 shutdown_connections 2024-02-14T17:46:06.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:06.413+0000 7f753292b700 1 -- 172.21.15.100:0/480599905 wait complete. 2024-02-14T17:46:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/933823346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T17:46:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:06 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:06 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/933823346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T17:46:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:06 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:06.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:06 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:06.780 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738417 got 34359738420 for osd.0 2024-02-14T17:46:06.780 DEBUG:teuthology.parallel:result is None 2024-02-14T17:46:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/933823346' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-14T17:46:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:06 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:06 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:07.527 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:07 smithi100 ceph-mon[39557]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:07.528 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/480599905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-14T17:46:07.528 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:07 smithi100 ceph-mon[30138]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:07.528 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/480599905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-14T17:46:07.529 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182435 got 107374182438 for osd.3 2024-02-14T17:46:07.529 DEBUG:teuthology.parallel:result is None 2024-02-14T17:46:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:07 smithi179 ceph-mon[33633]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/480599905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-14T17:46:08.729 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411371 for osd.2 2024-02-14T17:46:08.729 DEBUG:teuthology.parallel:result is None 2024-02-14T17:46:08.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.912+0000 7f9f5e224700 1 -- 172.21.15.100:0/1153622314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58107dc0 msgr2=0x7f9f581081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.912+0000 7f9f5e224700 1 --2- 172.21.15.100:0/1153622314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58107dc0 0x7f9f581081a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f480099c0 tx=0x7f9f48009cd0 comp rx=0 tx=0).stop 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.912+0000 7f9f5e224700 1 -- 172.21.15.100:0/1153622314 shutdown_connections 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.912+0000 7f9f5e224700 1 --2- 172.21.15.100:0/1153622314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58100b80 0x7f9f5810eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.912+0000 7f9f5e224700 1 --2- 172.21.15.100:0/1153622314 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58107dc0 0x7f9f581081a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.912+0000 7f9f5e224700 1 --2- 172.21.15.100:0/1153622314 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f581001c0 0x7f9f58100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.912+0000 7f9f5e224700 1 -- 172.21.15.100:0/1153622314 >> 172.21.15.100:0/1153622314 conn(0x7f9f58077140 msgr2=0x7f9f580fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.912+0000 7f9f5e224700 1 -- 172.21.15.100:0/1153622314 shutdown_connections 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.913+0000 7f9f5e224700 1 -- 172.21.15.100:0/1153622314 wait complete. 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.913+0000 7f9f5e224700 1 Processor -- start 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.913+0000 7f9f5e224700 1 -- start start 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f5e224700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f581001c0 0x7f9f5819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:08.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f5e224700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58100b80 0x7f9f5819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f5e224700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58107dc0 0x7f9f581976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f5e224700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58197e90 con 0x7f9f58107dc0 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f5e224700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58198000 con 0x7f9f58100b80 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f5e224700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581981c0 con 0x7f9f581001c0 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f57fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58107dc0 0x7f9f581976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f581001c0 0x7f9f5819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f577fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f581001c0 0x7f9f5819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:56712/0 (socket says 172.21.15.100:56712) 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f57fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58107dc0 0x7f9f581976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:38240/0 (socket says 172.21.15.100:38240) 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f577fe700 1 -- 172.21.15.100:0/3839856904 learned_addr learned my addr 172.21.15.100:0/3839856904 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f56ffd700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58100b80 0x7f9f5819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f57fff700 1 -- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f581001c0 msgr2=0x7f9f5819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f57fff700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f581001c0 0x7f9f5819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:08.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f57fff700 1 -- 172.21.15.100:0/3839856904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58100b80 msgr2=0x7f9f5819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f57fff700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58100b80 0x7f9f5819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f57fff700 1 -- 172.21.15.100:0/3839856904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c009520 con 0x7f9f58107dc0 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f56ffd700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58100b80 0x7f9f5819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f57fff700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58107dc0 0x7f9f581976e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000be20 tx=0x7f9f50009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f54ff9700 1 -- 172.21.15.100:0/3839856904 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000cc10 con 0x7f9f58107dc0 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f5e224700 1 -- 172.21.15.100:0/3839856904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f480095f0 con 0x7f9f58107dc0 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f54ff9700 1 -- 172.21.15.100:0/3839856904 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f9f5000cd70 con 0x7f9f58107dc0 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f54ff9700 1 -- 172.21.15.100:0/3839856904 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500125a0 con 0x7f9f58107dc0 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.914+0000 7f9f5e224700 1 -- 172.21.15.100:0/3839856904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f581a4590 con 0x7f9f58107dc0 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.915+0000 7f9f5e224700 1 -- 172.21.15.100:0/3839856904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f581034e0 con 0x7f9f58107dc0 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.916+0000 7f9f54ff9700 1 -- 172.21.15.100:0/3839856904 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f9f50014440 con 0x7f9f58107dc0 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.916+0000 7f9f54ff9700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f9f4006dac0 0x7f9f4006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.916+0000 7f9f54ff9700 1 -- 172.21.15.100:0/3839856904 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f9f50006e40 con 0x7f9f58107dc0 2024-02-14T17:46:08.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.916+0000 7f9f577fe700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f9f4006dac0 0x7f9f4006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:08.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.917+0000 7f9f577fe700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f9f4006dac0 0x7f9f4006ff80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f48000c00 tx=0x7f9f48005fb0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:08.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:08.921+0000 7f9f54ff9700 1 -- 172.21.15.100:0/3839856904 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f50056680 con 0x7f9f58107dc0 2024-02-14T17:46:09.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.122+0000 7f9f5e224700 1 -- 172.21.15.100:0/3839856904 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9f58198c10 con 0x7f9f58107dc0 2024-02-14T17:46:09.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.122+0000 7f9f54ff9700 1 -- 172.21.15.100:0/3839856904 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f50059d40 con 0x7f9f58107dc0 2024-02-14T17:46:09.124 INFO:teuthology.orchestra.run.smithi100.stdout:55834574898 2024-02-14T17:46:09.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f9f5e224700 1 -- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f9f4006dac0 msgr2=0x7f9f4006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:09.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f9f5e224700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f9f4006dac0 0x7f9f4006ff80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9f48000c00 tx=0x7f9f48005fb0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f9f5e224700 1 -- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58107dc0 msgr2=0x7f9f581976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:09.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f9f5e224700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58107dc0 0x7f9f581976e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000be20 tx=0x7f9f50009ed0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f9f5e224700 1 -- 172.21.15.100:0/3839856904 shutdown_connections 2024-02-14T17:46:09.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f9f5e224700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f9f4006dac0 0x7f9f4006ff80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f9f5e224700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9f58107dc0 0x7f9f581976e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f9f5e224700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9f581001c0 0x7f9f5819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f9f5e224700 1 --2- 172.21.15.100:0/3839856904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f58100b80 0x7f9f5819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f9f5e224700 1 -- 172.21.15.100:0/3839856904 >> 172.21.15.100:0/3839856904 conn(0x7f9f58077140 msgr2=0x7f9f58104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:09.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.126+0000 7f9f5e224700 1 -- 172.21.15.100:0/3839856904 shutdown_connections 2024-02-14T17:46:09.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.126+0000 7f9f5e224700 1 -- 172.21.15.100:0/3839856904 wait complete. 2024-02-14T17:46:09.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.122+0000 7f18d83c9700 1 -- 172.21.15.100:0/56077910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18d0100b70 msgr2=0x7f18d0100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:09.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.122+0000 7f18d83c9700 1 --2- 172.21.15.100:0/56077910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18d0100b70 0x7f18d0100ff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009990 tx=0x7f18c0009ca0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.122+0000 7f18d83c9700 1 -- 172.21.15.100:0/56077910 shutdown_connections 2024-02-14T17:46:09.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.122+0000 7f18d83c9700 1 --2- 172.21.15.100:0/56077910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18d0101530 0x7f18d010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.122+0000 7f18d83c9700 1 --2- 172.21.15.100:0/56077910 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18d0106b80 0x7f18d0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.122+0000 7f18d83c9700 1 --2- 172.21.15.100:0/56077910 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18d0100b70 0x7f18d0100ff0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.122+0000 7f18d83c9700 1 -- 172.21.15.100:0/56077910 >> 172.21.15.100:0/56077910 conn(0x7f18d0077aa0 msgr2=0x7f18d0077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.123+0000 7f18d83c9700 1 -- 172.21.15.100:0/56077910 shutdown_connections 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.123+0000 7f18d83c9700 1 -- 172.21.15.100:0/56077910 wait complete. 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.123+0000 7f18d83c9700 1 Processor -- start 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.123+0000 7f18d83c9700 1 -- start start 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d83c9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18d0100b70 0x7f18d0197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d83c9700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18d0101530 0x7f18d0197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d83c9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18d0106b80 0x7f18d019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d83c9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d019c550 con 0x7f18d0106b80 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d83c9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d019c6c0 con 0x7f18d0100b70 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d83c9700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d019c880 con 0x7f18d0101530 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d5964700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18d0101530 0x7f18d0197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d6966700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18d0106b80 0x7f18d019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:09.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d5964700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18d0101530 0x7f18d0197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:56728/0 (socket says 172.21.15.100:56728) 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d5964700 1 -- 172.21.15.100:0/3072415355 learned_addr learned my addr 172.21.15.100:0/3072415355 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d6165700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18d0100b70 0x7f18d0197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d5964700 1 -- 172.21.15.100:0/3072415355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18d0100b70 msgr2=0x7f18d0197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d5964700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18d0100b70 0x7f18d0197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d5964700 1 -- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18d0106b80 msgr2=0x7f18d019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d5964700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18d0106b80 0x7f18d019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d5964700 1 -- 172.21.15.100:0/3072415355 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc009520 con 0x7f18d0101530 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d6966700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18d0106b80 0x7f18d019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d5964700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18d0101530 0x7f18d0197bb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18c0005770 tx=0x7f18c00057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18c77fe700 1 -- 172.21.15.100:0/3072415355 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c001d070 con 0x7f18d0101530 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18c77fe700 1 -- 172.21.15.100:0/3072415355 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f18c000be20 con 0x7f18d0101530 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18c77fe700 1 -- 172.21.15.100:0/3072415355 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0022d30 con 0x7f18d0101530 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.124+0000 7f18d83c9700 1 -- 172.21.15.100:0/3072415355 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c00095f0 con 0x7f18d0101530 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.125+0000 7f18d83c9700 1 -- 172.21.15.100:0/3072415355 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d019cd80 con 0x7f18d0101530 2024-02-14T17:46:09.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.126+0000 7f18d83c9700 1 -- 172.21.15.100:0/3072415355 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d01021f0 con 0x7f18d0101530 2024-02-14T17:46:09.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.126+0000 7f18c77fe700 1 -- 172.21.15.100:0/3072415355 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f18c000b9a0 con 0x7f18d0101530 2024-02-14T17:46:09.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.127+0000 7f18c77fe700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f18bc06dac0 0x7f18bc06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:09.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.127+0000 7f18d6165700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f18bc06dac0 0x7f18bc06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:09.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.127+0000 7f18c77fe700 1 -- 172.21.15.100:0/3072415355 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f18c008ee00 con 0x7f18d0101530 2024-02-14T17:46:09.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.127+0000 7f18d6165700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f18bc06dac0 0x7f18bc06ff80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009d80 tx=0x7f18cc009260 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:09.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.132+0000 7f18c77fe700 1 -- 172.21.15.100:0/3072415355 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18c0058330 con 0x7f18d0101530 2024-02-14T17:46:09.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.331+0000 7f18d83c9700 1 -- 172.21.15.100:0/3072415355 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f18d0198a00 con 0x7f18d0101530 2024-02-14T17:46:09.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.331+0000 7f18c77fe700 1 -- 172.21.15.100:0/3072415355 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18c0027090 con 0x7f18d0101530 2024-02-14T17:46:09.333 INFO:teuthology.orchestra.run.smithi100.stdout:163208757276 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 -- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f18bc06dac0 msgr2=0x7f18bc06ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f18bc06dac0 0x7f18bc06ff80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009d80 tx=0x7f18cc009260 comp rx=0 tx=0).stop 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 -- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18d0101530 msgr2=0x7f18d0197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18d0101530 0x7f18d0197bb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18c0005770 tx=0x7f18c00057a0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 -- 172.21.15.100:0/3072415355 shutdown_connections 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18d0100b70 0x7f18d0197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f18bc06dac0 0x7f18bc06ff80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18d0106b80 0x7f18d019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 --2- 172.21.15.100:0/3072415355 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f18d0101530 0x7f18d0197bb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 -- 172.21.15.100:0/3072415355 >> 172.21.15.100:0/3072415355 conn(0x7f18d0077aa0 msgr2=0x7f18d00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:09.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 -- 172.21.15.100:0/3072415355 shutdown_connections 2024-02-14T17:46:09.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:09.334+0000 7f18d83c9700 1 -- 172.21.15.100:0/3072415355 wait complete. 2024-02-14T17:46:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:09 smithi100 ceph-mon[30138]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3839856904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T17:46:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3072415355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-14T17:46:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:09 smithi100 ceph-mon[39557]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3839856904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T17:46:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3072415355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-14T17:46:09.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:09 smithi179 ceph-mon[33633]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:09.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3839856904' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-14T17:46:09.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3072415355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-14T17:46:10.032 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574898 for osd.1 2024-02-14T17:46:10.032 DEBUG:teuthology.parallel:result is None 2024-02-14T17:46:10.043 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:46:10.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:46:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:10.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:10 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:10.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:10 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:10 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:10.834 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757272 got 163208757276 for osd.5 2024-02-14T17:46:10.834 DEBUG:teuthology.parallel:result is None 2024-02-14T17:46:11.318 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:46:11.896 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:46:12.120 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi100 to 3760M 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.121 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: Updating smithi179:/etc/ceph/ceph.conf 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.122 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi100 to 3760M 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: Updating smithi179:/etc/ceph/ceph.conf 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:12.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi100 to 3760M 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: Updating smithi100:/etc/ceph/ceph.conf 2024-02-14T17:46:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: Updating smithi179:/etc/ceph/ceph.conf 2024-02-14T17:46:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:12.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.835+0000 7f2cd62dc700 1 -- 172.21.15.100:0/557601843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd0101cc0 msgr2=0x7f2cd0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:12.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.835+0000 7f2cd62dc700 1 --2- 172.21.15.100:0/557601843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd0101cc0 0x7f2cd0102140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40099c0 tx=0x7f2cc4009cd0 comp rx=0 tx=0).stop 2024-02-14T17:46:12.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.836+0000 7f2cd62dc700 1 -- 172.21.15.100:0/557601843 shutdown_connections 2024-02-14T17:46:12.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.836+0000 7f2cd62dc700 1 --2- 172.21.15.100:0/557601843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cd0107ca0 0x7f2cd0108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:12.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.836+0000 7f2cd62dc700 1 --2- 172.21.15.100:0/557601843 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd0102680 0x7f2cd010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:12.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.836+0000 7f2cd62dc700 1 --2- 172.21.15.100:0/557601843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd0101cc0 0x7f2cd0102140 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:12.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.836+0000 7f2cd62dc700 1 -- 172.21.15.100:0/557601843 >> 172.21.15.100:0/557601843 conn(0x7f2cd0078da0 msgr2=0x7f2cd00791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:12.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.836+0000 7f2cd62dc700 1 -- 172.21.15.100:0/557601843 shutdown_connections 2024-02-14T17:46:12.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.836+0000 7f2cd62dc700 1 -- 172.21.15.100:0/557601843 wait complete. 2024-02-14T17:46:12.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.836+0000 7f2cd62dc700 1 Processor -- start 2024-02-14T17:46:12.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.836+0000 7f2cd62dc700 1 -- start start 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2cd62dc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd0101cc0 0x7f2cd0197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2cd62dc700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd0102680 0x7f2cd0197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2cd62dc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cd0107ca0 0x7f2cd019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2cd62dc700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd019c5e0 con 0x7f2cd0101cc0 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2cd62dc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd019c750 con 0x7f2cd0107ca0 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2cd62dc700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd019c910 con 0x7f2cd0102680 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd0101cc0 0x7f2cd0197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccf7fe700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd0102680 0x7f2cd0197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd0101cc0 0x7f2cd0197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58454/0 (socket says 172.21.15.100:58454) 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccffff700 1 -- 172.21.15.100:0/2082508022 learned_addr learned my addr 172.21.15.100:0/2082508022 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2cd4879700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cd0107ca0 0x7f2cd019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccf7fe700 1 -- 172.21.15.100:0/2082508022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cd0107ca0 msgr2=0x7f2cd019bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccf7fe700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cd0107ca0 0x7f2cd019bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccf7fe700 1 -- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd0101cc0 msgr2=0x7f2cd0197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccf7fe700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd0101cc0 0x7f2cd0197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccf7fe700 1 -- 172.21.15.100:0/2082508022 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc40095f0 con 0x7f2cd0102680 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccffff700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd0101cc0 0x7f2cd0197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2cd4879700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cd0107ca0 0x7f2cd019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccf7fe700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd0102680 0x7f2cd0197c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40098e0 tx=0x7f2cc400b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccd7fa700 1 -- 172.21.15.100:0/2082508022 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc401d070 con 0x7f2cd0102680 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2cd62dc700 1 -- 172.21.15.100:0/2082508022 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd019cb90 con 0x7f2cd0102680 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2ccd7fa700 1 -- 172.21.15.100:0/2082508022 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f2cc400bdf0 con 0x7f2cd0102680 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.837+0000 7f2cd62dc700 1 -- 172.21.15.100:0/2082508022 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd01a4450 con 0x7f2cd0102680 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.838+0000 7f2ccd7fa700 1 -- 172.21.15.100:0/2082508022 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400f970 con 0x7f2cd0102680 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.838+0000 7f2cd62dc700 1 -- 172.21.15.100:0/2082508022 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd010a180 con 0x7f2cd0102680 2024-02-14T17:46:12.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.839+0000 7f2ccd7fa700 1 -- 172.21.15.100:0/2082508022 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f2cc400fad0 con 0x7f2cd0102680 2024-02-14T17:46:12.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.839+0000 7f2ccd7fa700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f2cb806dac0 0x7f2cb806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:12.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.839+0000 7f2ccd7fa700 1 -- 172.21.15.100:0/2082508022 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f2cc408e5d0 con 0x7f2cd0102680 2024-02-14T17:46:12.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.840+0000 7f2ccffff700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f2cb806dac0 0x7f2cb806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:12.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.840+0000 7f2ccffff700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f2cb806dac0 0x7f2cb806ff80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005950 tx=0x7f2cc000b500 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:12.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:12.845+0000 7f2ccd7fa700 1 -- 172.21.15.100:0/2082508022 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2cc4057b70 con 0x7f2cd0102680 2024-02-14T17:46:13.082 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.080+0000 7f2cd62dc700 1 -- 172.21.15.100:0/2082508022 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2cd0068930 con 0x7f2cd0102680 2024-02-14T17:46:13.082 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[39557]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:46:13.083 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[39557]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:46:13.083 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[39557]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:46:13.083 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[39557]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:46:13.083 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:13.083 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[39557]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:46:13.083 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[39557]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:46:13.083 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:13.083 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:13.083 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[39557]: Deploying daemon alertmanager.a on smithi100 2024-02-14T17:46:13.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.081+0000 7f2ccd7fa700 1 -- 172.21.15.100:0/2082508022 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2cc405b230 con 0x7f2cd0102680 2024-02-14T17:46:13.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[30138]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:46:13.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[30138]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:46:13.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[30138]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:46:13.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[30138]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:46:13.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:13.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[30138]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:46:13.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[30138]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:46:13.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:13.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:13.084 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:12 smithi100 ceph-mon[30138]: Deploying daemon alertmanager.a on smithi100 2024-02-14T17:46:13.085 INFO:teuthology.orchestra.run.smithi100.stdout:197568495641 2024-02-14T17:46:13.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 -- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f2cb806dac0 msgr2=0x7f2cb806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:13.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f2cb806dac0 0x7f2cb806ff80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005950 tx=0x7f2cc000b500 comp rx=0 tx=0).stop 2024-02-14T17:46:13.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 -- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd0102680 msgr2=0x7f2cd0197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd0102680 0x7f2cd0197c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40098e0 tx=0x7f2cc400b9c0 comp rx=0 tx=0).stop 2024-02-14T17:46:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 -- 172.21.15.100:0/2082508022 shutdown_connections 2024-02-14T17:46:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2cd0107ca0 0x7f2cd019bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f2cb806dac0 0x7f2cb806ff80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cd0101cc0 0x7f2cd0197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 --2- 172.21.15.100:0/2082508022 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f2cd0102680 0x7f2cd0197c40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 -- 172.21.15.100:0/2082508022 >> 172.21.15.100:0/2082508022 conn(0x7f2cd0078da0 msgr2=0x7f2cd0100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 -- 172.21.15.100:0/2082508022 shutdown_connections 2024-02-14T17:46:13.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:13.083+0000 7f2cd62dc700 1 -- 172.21.15.100:0/2082508022 wait complete. 2024-02-14T17:46:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:12 smithi179 ceph-mon[33633]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:46:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:12 smithi179 ceph-mon[33633]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-14T17:46:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:12 smithi179 ceph-mon[33633]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:46:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:12 smithi179 ceph-mon[33633]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-02-14T17:46:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:12 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:12 smithi179 ceph-mon[33633]: Updating smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:46:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:12 smithi179 ceph-mon[33633]: Updating smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-14T17:46:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:12 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:12 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:12 smithi179 ceph-mon[33633]: Deploying daemon alertmanager.a on smithi100 2024-02-14T17:46:14.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:13 smithi100 ceph-mon[39557]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:14.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2082508022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-14T17:46:14.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:13 smithi100 ceph-mon[30138]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:14.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2082508022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-14T17:46:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:13 smithi179 ceph-mon[33633]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2082508022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-14T17:46:14.840 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495638 got 197568495641 for osd.6 2024-02-14T17:46:14.841 DEBUG:teuthology.parallel:result is None 2024-02-14T17:46:15.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.459+0000 7f53ee1c6700 1 -- 172.21.15.100:0/4157971103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e81026b0 msgr2=0x7f53e810eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:15.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.459+0000 7f53ee1c6700 1 --2- 172.21.15.100:0/4157971103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e81026b0 0x7f53e810eb10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009f90 tx=0x7f53e000b330 comp rx=0 tx=0).stop 2024-02-14T17:46:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.459+0000 7f53ee1c6700 1 -- 172.21.15.100:0/4157971103 shutdown_connections 2024-02-14T17:46:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.459+0000 7f53ee1c6700 1 --2- 172.21.15.100:0/4157971103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e8107dd0 0x7f53e81081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.459+0000 7f53ee1c6700 1 --2- 172.21.15.100:0/4157971103 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e81026b0 0x7f53e810eb10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.459+0000 7f53ee1c6700 1 --2- 172.21.15.100:0/4157971103 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e8101cf0 0x7f53e8102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.459+0000 7f53ee1c6700 1 -- 172.21.15.100:0/4157971103 >> 172.21.15.100:0/4157971103 conn(0x7f53e8078d30 msgr2=0x7f53e8079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.459+0000 7f53ee1c6700 1 -- 172.21.15.100:0/4157971103 shutdown_connections 2024-02-14T17:46:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.459+0000 7f53ee1c6700 1 -- 172.21.15.100:0/4157971103 wait complete. 2024-02-14T17:46:15.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.459+0000 7f53ee1c6700 1 Processor -- start 2024-02-14T17:46:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53ee1c6700 1 -- start start 2024-02-14T17:46:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53ee1c6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e8101cf0 0x7f53e819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53ee1c6700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e81026b0 0x7f53e819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53ee1c6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e8107dd0 0x7f53e81975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53ee1c6700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e8197c50 con 0x7f53e8107dd0 2024-02-14T17:46:15.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53ee1c6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e8197d90 con 0x7f53e8101cf0 2024-02-14T17:46:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53ee1c6700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e8197f50 con 0x7f53e81026b0 2024-02-14T17:46:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e8107dd0 0x7f53e81975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e6ffd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e81026b0 0x7f53e819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e7fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e8107dd0 0x7f53e81975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58476/0 (socket says 172.21.15.100:58476) 2024-02-14T17:46:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e7fff700 1 -- 172.21.15.100:0/1488499786 learned_addr learned my addr 172.21.15.100:0/1488499786 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:46:15.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e77fe700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e8101cf0 0x7f53e819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:15.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e7fff700 1 -- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e81026b0 msgr2=0x7f53e819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:15.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e7fff700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e81026b0 0x7f53e819d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:15.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e7fff700 1 -- 172.21.15.100:0/1488499786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e8101cf0 msgr2=0x7f53e819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:15.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e7fff700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e8101cf0 0x7f53e819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:15.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e7fff700 1 -- 172.21.15.100:0/1488499786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc0095f0 con 0x7f53e8107dd0 2024-02-14T17:46:15.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e6ffd700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e81026b0 0x7f53e819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:46:15.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.460+0000 7f53e7fff700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e8107dd0 0x7f53e81975f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f53e000b6c0 tx=0x7f53e000bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:15.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.461+0000 7f53e4ff9700 1 -- 172.21.15.100:0/1488499786 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e000e040 con 0x7f53e8107dd0 2024-02-14T17:46:15.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.461+0000 7f53ee1c6700 1 -- 172.21.15.100:0/1488499786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e0009c40 con 0x7f53e8107dd0 2024-02-14T17:46:15.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.461+0000 7f53e4ff9700 1 -- 172.21.15.100:0/1488499786 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f53e0004e30 con 0x7f53e8107dd0 2024-02-14T17:46:15.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.461+0000 7f53e4ff9700 1 -- 172.21.15.100:0/1488499786 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e001b6b0 con 0x7f53e8107dd0 2024-02-14T17:46:15.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.461+0000 7f53ee1c6700 1 -- 172.21.15.100:0/1488499786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e81a4300 con 0x7f53e8107dd0 2024-02-14T17:46:15.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.462+0000 7f53ee1c6700 1 -- 172.21.15.100:0/1488499786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e810a260 con 0x7f53e8107dd0 2024-02-14T17:46:15.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.462+0000 7f53e4ff9700 1 -- 172.21.15.100:0/1488499786 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f53e001d480 con 0x7f53e8107dd0 2024-02-14T17:46:15.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.463+0000 7f53e4ff9700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f53d006dac0 0x7f53d006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:15.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.463+0000 7f53e77fe700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f53d006dac0 0x7f53d006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:15.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.463+0000 7f53e4ff9700 1 -- 172.21.15.100:0/1488499786 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f53e008f000 con 0x7f53e8107dd0 2024-02-14T17:46:15.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.463+0000 7f53e77fe700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f53d006dac0 0x7f53d006ff80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53d8005950 tx=0x7f53d800a300 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:15.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.468+0000 7f53e4ff9700 1 -- 172.21.15.100:0/1488499786 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53e0058410 con 0x7f53e8107dd0 2024-02-14T17:46:15.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.698+0000 7f53ee1c6700 1 -- 172.21.15.100:0/1488499786 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f53e8198b20 con 0x7f53e8107dd0 2024-02-14T17:46:15.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.698+0000 7f53e4ff9700 1 -- 172.21.15.100:0/1488499786 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53e005bad0 con 0x7f53e8107dd0 2024-02-14T17:46:15.700 INFO:teuthology.orchestra.run.smithi100.stdout:128849018913 2024-02-14T17:46:15.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.700+0000 7f53ee1c6700 1 -- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f53d006dac0 msgr2=0x7f53d006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:15.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.700+0000 7f53ee1c6700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f53d006dac0 0x7f53d006ff80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53d8005950 tx=0x7f53d800a300 comp rx=0 tx=0).stop 2024-02-14T17:46:15.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.700+0000 7f53ee1c6700 1 -- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e8107dd0 msgr2=0x7f53e81975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:15.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.700+0000 7f53ee1c6700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e8107dd0 0x7f53e81975f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f53e000b6c0 tx=0x7f53e000bc40 comp rx=0 tx=0).stop 2024-02-14T17:46:15.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.700+0000 7f53ee1c6700 1 -- 172.21.15.100:0/1488499786 shutdown_connections 2024-02-14T17:46:15.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.700+0000 7f53ee1c6700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53e8101cf0 0x7f53e819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:15.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.700+0000 7f53ee1c6700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f53d006dac0 0x7f53d006ff80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:15.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.700+0000 7f53ee1c6700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53e8107dd0 0x7f53e81975f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:15.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.700+0000 7f53ee1c6700 1 --2- 172.21.15.100:0/1488499786 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f53e81026b0 0x7f53e819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:15.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.700+0000 7f53ee1c6700 1 -- 172.21.15.100:0/1488499786 >> 172.21.15.100:0/1488499786 conn(0x7f53e8078d30 msgr2=0x7f53e8100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:15.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.701+0000 7f53ee1c6700 1 -- 172.21.15.100:0/1488499786 shutdown_connections 2024-02-14T17:46:15.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:15.701+0000 7f53ee1c6700 1 -- 172.21.15.100:0/1488499786 wait complete. 2024-02-14T17:46:16.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:15 smithi100 ceph-mon[30138]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:16.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1488499786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-14T17:46:16.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:15 smithi100 ceph-mon[39557]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:16.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1488499786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-14T17:46:16.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:15 smithi179 ceph-mon[33633]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:16.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1488499786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-14T17:46:17.087 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018909 got 128849018913 for osd.4 2024-02-14T17:46:17.088 DEBUG:teuthology.parallel:result is None 2024-02-14T17:46:17.088 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-14T17:46:17.088 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-14T17:46:17.343 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:46:18.238 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:17 smithi100 ceph-mon[39557]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:18.238 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:17 smithi100 ceph-mon[30138]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:17 smithi179 ceph-mon[33633]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:20.071 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:19 smithi100 ceph-mon[39557]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:20.072 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:19 smithi100 ceph-mon[30138]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:20.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.309+0000 7faf37fb2700 1 -- 172.21.15.100:0/3715702073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf301011b0 msgr2=0x7faf301068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:20.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.309+0000 7faf37fb2700 1 --2- 172.21.15.100:0/3715702073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf301011b0 0x7faf301068f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0098f0 tx=0x7faf2c009c00 comp rx=0 tx=0).stop 2024-02-14T17:46:20.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.309+0000 7faf37fb2700 1 -- 172.21.15.100:0/3715702073 shutdown_connections 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.309+0000 7faf37fb2700 1 --2- 172.21.15.100:0/3715702073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30100800 0x7faf30100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.309+0000 7faf37fb2700 1 --2- 172.21.15.100:0/3715702073 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf30107020 0x7faf3010eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.309+0000 7faf37fb2700 1 --2- 172.21.15.100:0/3715702073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf301011b0 0x7faf301068f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.309+0000 7faf37fb2700 1 -- 172.21.15.100:0/3715702073 >> 172.21.15.100:0/3715702073 conn(0x7faf30075ea0 msgr2=0x7faf300762b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf37fb2700 1 -- 172.21.15.100:0/3715702073 shutdown_connections 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf37fb2700 1 -- 172.21.15.100:0/3715702073 wait complete. 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf37fb2700 1 Processor -- start 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf37fb2700 1 -- start start 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf37fb2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30100800 0x7faf30101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf37fb2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf301011b0 0x7faf30101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf37fb2700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf30107020 0x7faf30102200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf37fb2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf301a5bf0 con 0x7faf301011b0 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf37fb2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf301a5d60 con 0x7faf30100800 2024-02-14T17:46:20.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf37fb2700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf301a5f20 con 0x7faf30107020 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf3554d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf301011b0 0x7faf30101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.310+0000 7faf3654f700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf30107020 0x7faf30102200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf3554d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf301011b0 0x7faf30101cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:58488/0 (socket says 172.21.15.100:58488) 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf3554d700 1 -- 172.21.15.100:0/3829182190 learned_addr learned my addr 172.21.15.100:0/3829182190 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf35d4e700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30100800 0x7faf30101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf3554d700 1 -- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf30107020 msgr2=0x7faf30102200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf3554d700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf30107020 0x7faf30102200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf3554d700 1 -- 172.21.15.100:0/3829182190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30100800 msgr2=0x7faf30101780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf3554d700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30100800 0x7faf30101780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf3554d700 1 -- 172.21.15.100:0/3829182190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf28009d00 con 0x7faf301011b0 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf3654f700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf30107020 0x7faf30102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf3554d700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf301011b0 0x7faf30101cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009230 tx=0x7faf2c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf26ffd700 1 -- 172.21.15.100:0/3829182190 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c01c070 con 0x7faf301011b0 2024-02-14T17:46:20.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf26ffd700 1 -- 172.21.15.100:0/3829182190 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7faf2c00bae0 con 0x7faf301011b0 2024-02-14T17:46:20.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf37fb2700 1 -- 172.21.15.100:0/3829182190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c009520 con 0x7faf301011b0 2024-02-14T17:46:20.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf26ffd700 1 -- 172.21.15.100:0/3829182190 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0038e0 con 0x7faf301011b0 2024-02-14T17:46:20.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.311+0000 7faf37fb2700 1 -- 172.21.15.100:0/3829182190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf301a6420 con 0x7faf301011b0 2024-02-14T17:46:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.312+0000 7faf37fb2700 1 -- 172.21.15.100:0/3829182190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf30068930 con 0x7faf301011b0 2024-02-14T17:46:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.313+0000 7faf26ffd700 1 -- 172.21.15.100:0/3829182190 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7faf2c020030 con 0x7faf301011b0 2024-02-14T17:46:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.313+0000 7faf26ffd700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7faf1c06dac0 0x7faf1c06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.313+0000 7faf35d4e700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7faf1c06dac0 0x7faf1c06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.313+0000 7faf26ffd700 1 -- 172.21.15.100:0/3829182190 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7faf2c08e890 con 0x7faf301011b0 2024-02-14T17:46:20.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.313+0000 7faf35d4e700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7faf1c06dac0 0x7faf1c06ff80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf20006fd0 tx=0x7faf20008040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:20.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.318+0000 7faf26ffd700 1 -- 172.21.15.100:0/3829182190 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf2c057ca0 con 0x7faf301011b0 2024-02-14T17:46:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:19 smithi179 ceph-mon[33633]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:20.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.516+0000 7faf37fb2700 1 -- 172.21.15.100:0/3829182190 --> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf30111250 con 0x7faf1c06dac0 2024-02-14T17:46:20.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.528+0000 7faf26ffd700 1 -- 172.21.15.100:0/3829182190 <== mgr.24482 v2:172.21.15.100:6800/1773294710 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+383774 (secure 0 0 0) 0x7faf30111250 con 0x7faf1c06dac0 2024-02-14T17:46:20.535 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:46:20.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 -- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7faf1c06dac0 msgr2=0x7faf1c06ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7faf1c06dac0 0x7faf1c06ff80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf20006fd0 tx=0x7faf20008040 comp rx=0 tx=0).stop 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 -- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf301011b0 msgr2=0x7faf30101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf301011b0 0x7faf30101cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009230 tx=0x7faf2c00fa00 comp rx=0 tx=0).stop 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 -- 172.21.15.100:0/3829182190 shutdown_connections 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf30100800 0x7faf30101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7faf1c06dac0 0x7faf1c06ff80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7faf30107020 0x7faf30102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 --2- 172.21.15.100:0/3829182190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf301011b0 0x7faf30101cc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 -- 172.21.15.100:0/3829182190 >> 172.21.15.100:0/3829182190 conn(0x7faf30075ea0 msgr2=0x7faf300fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 -- 172.21.15.100:0/3829182190 shutdown_connections 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:20.531+0000 7faf37fb2700 1 -- 172.21.15.100:0/3829182190 wait complete. 2024-02-14T17:46:20.542 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-02-14T17:46:20.773 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:46:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:21.914 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:46:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:46:22.107 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:21 smithi100 ceph-mon[39557]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:22.108 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:21 smithi100 ceph-mon[39557]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:46:22.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:21 smithi100 ceph-mon[30138]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:22.108 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:21 smithi100 ceph-mon[30138]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:46:22.141 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-02-14T17:46:20.240444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1003,"num_read_kb":849,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1771654,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":236724,"kb_used_data":8092,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749724492,"statfs":{"total":767960285184,"available":767717879808,"internally_reserved":0,"allocated":8286208,"data_stored":4480988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002953"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222466+0000","last_change":"2024-02-14T17:45:04.714625+0000","last_active":"2024-02-14T17:45:40.222466+0000","last_peered":"2024-02-14T17:45:40.222466+0000","last_clean":"2024-02-14T17:45:40.222466+0000","last_became_active":"2024-02-14T17:45:04.714309+0000","last_became_peered":"2024-02-14T17:45:04.714309+0000","last_unstale":"2024-02-14T17:45:40.222466+0000","last_undegraded":"2024-02-14T17:45:40.222466+0000","last_fullsized":"2024-02-14T17:45:40.222466+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:35:23.162459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223388+0000","last_change":"2024-02-14T17:45:02.703191+0000","last_active":"2024-02-14T17:45:40.223388+0000","last_peered":"2024-02-14T17:45:40.223388+0000","last_clean":"2024-02-14T17:45:40.223388+0000","last_became_active":"2024-02-14T17:45:02.702872+0000","last_became_peered":"2024-02-14T17:45:02.702872+0000","last_unstale":"2024-02-14T17:45:40.223388+0000","last_undegraded":"2024-02-14T17:45:40.223388+0000","last_fullsized":"2024-02-14T17:45:40.223388+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:49:15.006665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551252+0000","last_change":"2024-02-14T17:44:58.678059+0000","last_active":"2024-02-14T17:45:40.551252+0000","last_peered":"2024-02-14T17:45:40.551252+0000","last_clean":"2024-02-14T17:45:40.551252+0000","last_became_active":"2024-02-14T17:44:58.677799+0000","last_became_peered":"2024-02-14T17:44:58.677799+0000","last_unstale":"2024-02-14T17:45:40.551252+0000","last_undegraded":"2024-02-14T17:45:40.551252+0000","last_fullsized":"2024-02-14T17:45:40.551252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:37:25.833721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222312+0000","last_change":"2024-02-14T17:45:00.678177+0000","last_active":"2024-02-14T17:45:40.222312+0000","last_peered":"2024-02-14T17:45:40.222312+0000","last_clean":"2024-02-14T17:45:40.222312+0000","last_became_active":"2024-02-14T17:45:00.677808+0000","last_became_peered":"2024-02-14T17:45:00.677808+0000","last_unstale":"2024-02-14T17:45:40.222312+0000","last_undegraded":"2024-02-14T17:45:40.222312+0000","last_fullsized":"2024-02-14T17:45:40.222312+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:43:40.205055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551788+0000","last_change":"2024-02-14T17:45:04.950149+0000","last_active":"2024-02-14T17:45:40.551788+0000","last_peered":"2024-02-14T17:45:40.551788+0000","last_clean":"2024-02-14T17:45:40.551788+0000","last_became_active":"2024-02-14T17:45:04.949753+0000","last_became_peered":"2024-02-14T17:45:04.949753+0000","last_unstale":"2024-02-14T17:45:40.551788+0000","last_undegraded":"2024-02-14T17:45:40.551788+0000","last_fullsized":"2024-02-14T17:45:40.551788+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:51:31.912410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222319+0000","last_change":"2024-02-14T17:45:02.696717+0000","last_active":"2024-02-14T17:45:40.222319+0000","last_peered":"2024-02-14T17:45:40.222319+0000","last_clean":"2024-02-14T17:45:40.222319+0000","last_became_active":"2024-02-14T17:45:02.696221+0000","last_became_peered":"2024-02-14T17:45:02.696221+0000","last_unstale":"2024-02-14T17:45:40.222319+0000","last_undegraded":"2024-02-14T17:45:40.222319+0000","last_fullsized":"2024-02-14T17:45:40.222319+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:28:48.835710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"61'1","reported_seq":39,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552583+0000","last_change":"2024-02-14T17:44:58.670983+0000","last_active":"2024-02-14T17:45:40.552583+0000","last_peered":"2024-02-14T17:45:40.552583+0000","last_clean":"2024-02-14T17:45:40.552583+0000","last_became_active":"2024-02-14T17:44:58.670683+0000","last_became_peered":"2024-02-14T17:44:58.670683+0000","last_unstale":"2024-02-14T17:45:40.552583+0000","last_undegraded":"2024-02-14T17:45:40.552583+0000","last_fullsized":"2024-02-14T17:45:40.552583+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:47:58.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551320+0000","last_change":"2024-02-14T17:45:00.686822+0000","last_active":"2024-02-14T17:45:40.551320+0000","last_peered":"2024-02-14T17:45:40.551320+0000","last_clean":"2024-02-14T17:45:40.551320+0000","last_became_active":"2024-02-14T17:45:00.686499+0000","last_became_peered":"2024-02-14T17:45:00.686499+0000","last_unstale":"2024-02-14T17:45:40.551320+0000","last_undegraded":"2024-02-14T17:45:40.551320+0000","last_fullsized":"2024-02-14T17:45:40.551320+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:02:08.567402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550597+0000","last_change":"2024-02-14T17:45:04.718285+0000","last_active":"2024-02-14T17:45:40.550597+0000","last_peered":"2024-02-14T17:45:40.550597+0000","last_clean":"2024-02-14T17:45:40.550597+0000","last_became_active":"2024-02-14T17:45:04.717880+0000","last_became_peered":"2024-02-14T17:45:04.717880+0000","last_unstale":"2024-02-14T17:45:40.550597+0000","last_undegraded":"2024-02-14T17:45:40.550597+0000","last_fullsized":"2024-02-14T17:45:40.550597+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:58:29.490294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332545+0000","last_change":"2024-02-14T17:45:02.698551+0000","last_active":"2024-02-14T17:45:40.332545+0000","last_peered":"2024-02-14T17:45:40.332545+0000","last_clean":"2024-02-14T17:45:40.332545+0000","last_became_active":"2024-02-14T17:45:02.698226+0000","last_became_peered":"2024-02-14T17:45:02.698226+0000","last_unstale":"2024-02-14T17:45:40.332545+0000","last_undegraded":"2024-02-14T17:45:40.332545+0000","last_fullsized":"2024-02-14T17:45:40.332545+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:28:24.789299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332744+0000","last_change":"2024-02-14T17:44:58.672885+0000","last_active":"2024-02-14T17:45:40.332744+0000","last_peered":"2024-02-14T17:45:40.332744+0000","last_clean":"2024-02-14T17:45:40.332744+0000","last_became_active":"2024-02-14T17:44:58.672565+0000","last_became_peered":"2024-02-14T17:44:58.672565+0000","last_unstale":"2024-02-14T17:45:40.332744+0000","last_undegraded":"2024-02-14T17:45:40.332744+0000","last_fullsized":"2024-02-14T17:45:40.332744+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:34:41.155588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222830+0000","last_change":"2024-02-14T17:45:00.688304+0000","last_active":"2024-02-14T17:45:40.222830+0000","last_peered":"2024-02-14T17:45:40.222830+0000","last_clean":"2024-02-14T17:45:40.222830+0000","last_became_active":"2024-02-14T17:45:00.687943+0000","last_became_peered":"2024-02-14T17:45:00.687943+0000","last_unstale":"2024-02-14T17:45:40.222830+0000","last_undegraded":"2024-02-14T17:45:40.222830+0000","last_fullsized":"2024-02-14T17:45:40.222830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:50:05.670902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332278+0000","last_change":"2024-02-14T17:45:04.708369+0000","last_active":"2024-02-14T17:45:40.332278+0000","last_peered":"2024-02-14T17:45:40.332278+0000","last_clean":"2024-02-14T17:45:40.332278+0000","last_became_active":"2024-02-14T17:45:04.708036+0000","last_became_peered":"2024-02-14T17:45:04.708036+0000","last_unstale":"2024-02-14T17:45:40.332278+0000","last_undegraded":"2024-02-14T17:45:40.332278+0000","last_fullsized":"2024-02-14T17:45:40.332278+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:19:20.045807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228087+0000","last_change":"2024-02-14T17:45:02.706149+0000","last_active":"2024-02-14T17:45:40.228087+0000","last_peered":"2024-02-14T17:45:40.228087+0000","last_clean":"2024-02-14T17:45:40.228087+0000","last_became_active":"2024-02-14T17:45:02.705784+0000","last_became_peered":"2024-02-14T17:45:02.705784+0000","last_unstale":"2024-02-14T17:45:40.228087+0000","last_undegraded":"2024-02-14T17:45:40.228087+0000","last_fullsized":"2024-02-14T17:45:40.228087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:41:27.847352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332616+0000","last_change":"2024-02-14T17:44:58.674526+0000","last_active":"2024-02-14T17:45:40.332616+0000","last_peered":"2024-02-14T17:45:40.332616+0000","last_clean":"2024-02-14T17:45:40.332616+0000","last_became_active":"2024-02-14T17:44:58.674246+0000","last_became_peered":"2024-02-14T17:44:58.674246+0000","last_unstale":"2024-02-14T17:45:40.332616+0000","last_undegraded":"2024-02-14T17:45:40.332616+0000","last_fullsized":"2024-02-14T17:45:40.332616+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:28:11.075624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":51,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223064+0000","last_change":"2024-02-14T17:45:19.847889+0000","last_active":"2024-02-14T17:45:40.223064+0000","last_peered":"2024-02-14T17:45:40.223064+0000","last_clean":"2024-02-14T17:45:40.223064+0000","last_became_active":"2024-02-14T17:45:19.843251+0000","last_became_peered":"2024-02-14T17:45:19.843251+0000","last_unstale":"2024-02-14T17:45:40.223064+0000","last_undegraded":"2024-02-14T17:45:40.223064+0000","last_fullsized":"2024-02-14T17:45:40.223064+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:29:56.479700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223436+0000","last_change":"2024-02-14T17:45:04.948373+0000","last_active":"2024-02-14T17:45:40.223436+0000","last_peered":"2024-02-14T17:45:40.223436+0000","last_clean":"2024-02-14T17:45:40.223436+0000","last_became_active":"2024-02-14T17:45:04.948046+0000","last_became_peered":"2024-02-14T17:45:04.948046+0000","last_unstale":"2024-02-14T17:45:40.223436+0000","last_undegraded":"2024-02-14T17:45:40.223436+0000","last_fullsized":"2024-02-14T17:45:40.223436+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:44:07.351791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224063+0000","last_change":"2024-02-14T17:45:02.702711+0000","last_active":"2024-02-14T17:45:40.224063+0000","last_peered":"2024-02-14T17:45:40.224063+0000","last_clean":"2024-02-14T17:45:40.224063+0000","last_became_active":"2024-02-14T17:45:02.702275+0000","last_became_peered":"2024-02-14T17:45:02.702275+0000","last_unstale":"2024-02-14T17:45:40.224063+0000","last_undegraded":"2024-02-14T17:45:40.224063+0000","last_fullsized":"2024-02-14T17:45:40.224063+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:05:52.488550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"61'1","reported_seq":39,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551787+0000","last_change":"2024-02-14T17:44:58.674923+0000","last_active":"2024-02-14T17:45:40.551787+0000","last_peered":"2024-02-14T17:45:40.551787+0000","last_clean":"2024-02-14T17:45:40.551787+0000","last_became_active":"2024-02-14T17:44:58.674588+0000","last_became_peered":"2024-02-14T17:44:58.674588+0000","last_unstale":"2024-02-14T17:45:40.551787+0000","last_undegraded":"2024-02-14T17:45:40.551787+0000","last_fullsized":"2024-02-14T17:45:40.551787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:29:47.449974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222168+0000","last_change":"2024-02-14T17:45:00.686532+0000","last_active":"2024-02-14T17:45:40.222168+0000","last_peered":"2024-02-14T17:45:40.222168+0000","last_clean":"2024-02-14T17:45:40.222168+0000","last_became_active":"2024-02-14T17:45:00.686090+0000","last_became_peered":"2024-02-14T17:45:00.686090+0000","last_unstale":"2024-02-14T17:45:40.222168+0000","last_undegraded":"2024-02-14T17:45:40.222168+0000","last_fullsized":"2024-02-14T17:45:40.222168+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:29:39.498895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552784+0000","last_change":"2024-02-14T17:45:04.950750+0000","last_active":"2024-02-14T17:45:40.552784+0000","last_peered":"2024-02-14T17:45:40.552784+0000","last_clean":"2024-02-14T17:45:40.552784+0000","last_became_active":"2024-02-14T17:45:04.948176+0000","last_became_peered":"2024-02-14T17:45:04.948176+0000","last_unstale":"2024-02-14T17:45:40.552784+0000","last_undegraded":"2024-02-14T17:45:40.552784+0000","last_fullsized":"2024-02-14T17:45:40.552784+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:54.998353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222839+0000","last_change":"2024-02-14T17:45:02.699200+0000","last_active":"2024-02-14T17:45:40.222839+0000","last_peered":"2024-02-14T17:45:40.222839+0000","last_clean":"2024-02-14T17:45:40.222839+0000","last_became_active":"2024-02-14T17:45:02.698894+0000","last_became_peered":"2024-02-14T17:45:02.698894+0000","last_unstale":"2024-02-14T17:45:40.222839+0000","last_undegraded":"2024-02-14T17:45:40.222839+0000","last_fullsized":"2024-02-14T17:45:40.222839+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:30:48.740310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222958+0000","last_change":"2024-02-14T17:44:58.671479+0000","last_active":"2024-02-14T17:45:40.222958+0000","last_peered":"2024-02-14T17:45:40.222958+0000","last_clean":"2024-02-14T17:45:40.222958+0000","last_became_active":"2024-02-14T17:44:58.671120+0000","last_became_peered":"2024-02-14T17:44:58.671120+0000","last_unstale":"2024-02-14T17:45:40.222958+0000","last_undegraded":"2024-02-14T17:45:40.222958+0000","last_fullsized":"2024-02-14T17:45:40.222958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:06:31.844279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552367+0000","last_change":"2024-02-14T17:45:00.685354+0000","last_active":"2024-02-14T17:45:40.552367+0000","last_peered":"2024-02-14T17:45:40.552367+0000","last_clean":"2024-02-14T17:45:40.552367+0000","last_became_active":"2024-02-14T17:45:00.684930+0000","last_became_peered":"2024-02-14T17:45:00.684930+0000","last_unstale":"2024-02-14T17:45:40.552367+0000","last_undegraded":"2024-02-14T17:45:40.552367+0000","last_fullsized":"2024-02-14T17:45:40.552367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:12:13.148305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228387+0000","last_change":"2024-02-14T17:45:04.713180+0000","last_active":"2024-02-14T17:45:40.228387+0000","last_peered":"2024-02-14T17:45:40.228387+0000","last_clean":"2024-02-14T17:45:40.228387+0000","last_became_active":"2024-02-14T17:45:04.712861+0000","last_became_peered":"2024-02-14T17:45:04.712861+0000","last_unstale":"2024-02-14T17:45:40.228387+0000","last_undegraded":"2024-02-14T17:45:40.228387+0000","last_fullsized":"2024-02-14T17:45:40.228387+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:13:47.945433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:08.731835+0000","last_change":"2024-02-14T17:45:02.699995+0000","last_active":"2024-02-14T17:46:08.731835+0000","last_peered":"2024-02-14T17:46:08.731835+0000","last_clean":"2024-02-14T17:46:08.731835+0000","last_became_active":"2024-02-14T17:45:02.699599+0000","last_became_peered":"2024-02-14T17:45:02.699599+0000","last_unstale":"2024-02-14T17:46:08.731835+0000","last_undegraded":"2024-02-14T17:46:08.731835+0000","last_fullsized":"2024-02-14T17:46:08.731835+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:12:23.842221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"61'1","reported_seq":34,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224986+0000","last_change":"2024-02-14T17:44:58.672652+0000","last_active":"2024-02-14T17:45:40.224986+0000","last_peered":"2024-02-14T17:45:40.224986+0000","last_clean":"2024-02-14T17:45:40.224986+0000","last_became_active":"2024-02-14T17:44:58.672328+0000","last_became_peered":"2024-02-14T17:44:58.672328+0000","last_unstale":"2024-02-14T17:45:40.224986+0000","last_undegraded":"2024-02-14T17:45:40.224986+0000","last_fullsized":"2024-02-14T17:45:40.224986+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:00.681939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551181+0000","last_change":"2024-02-14T17:45:00.688295+0000","last_active":"2024-02-14T17:45:40.551181+0000","last_peered":"2024-02-14T17:45:40.551181+0000","last_clean":"2024-02-14T17:45:40.551181+0000","last_became_active":"2024-02-14T17:45:00.687983+0000","last_became_peered":"2024-02-14T17:45:00.687983+0000","last_unstale":"2024-02-14T17:45:40.551181+0000","last_undegraded":"2024-02-14T17:45:40.551181+0000","last_fullsized":"2024-02-14T17:45:40.551181+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:30:49.810191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224199+0000","last_change":"2024-02-14T17:45:04.712972+0000","last_active":"2024-02-14T17:45:40.224199+0000","last_peered":"2024-02-14T17:45:40.224199+0000","last_clean":"2024-02-14T17:45:40.224199+0000","last_became_active":"2024-02-14T17:45:04.712646+0000","last_became_peered":"2024-02-14T17:45:04.712646+0000","last_unstale":"2024-02-14T17:45:40.224199+0000","last_undegraded":"2024-02-14T17:45:40.224199+0000","last_fullsized":"2024-02-14T17:45:40.224199+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:30:48.679915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551457+0000","last_change":"2024-02-14T17:45:02.705091+0000","last_active":"2024-02-14T17:45:40.551457+0000","last_peered":"2024-02-14T17:45:40.551457+0000","last_clean":"2024-02-14T17:45:40.551457+0000","last_became_active":"2024-02-14T17:45:02.704276+0000","last_became_peered":"2024-02-14T17:45:02.704276+0000","last_unstale":"2024-02-14T17:45:40.551457+0000","last_undegraded":"2024-02-14T17:45:40.551457+0000","last_fullsized":"2024-02-14T17:45:40.551457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:09:09.959532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550918+0000","last_change":"2024-02-14T17:44:58.675493+0000","last_active":"2024-02-14T17:45:40.550918+0000","last_peered":"2024-02-14T17:45:40.550918+0000","last_clean":"2024-02-14T17:45:40.550918+0000","last_became_active":"2024-02-14T17:44:58.673635+0000","last_became_peered":"2024-02-14T17:44:58.673635+0000","last_unstale":"2024-02-14T17:45:40.550918+0000","last_undegraded":"2024-02-14T17:45:40.550918+0000","last_fullsized":"2024-02-14T17:45:40.550918+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:21.539711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550907+0000","last_change":"2024-02-14T17:45:00.680999+0000","last_active":"2024-02-14T17:45:40.550907+0000","last_peered":"2024-02-14T17:45:40.550907+0000","last_clean":"2024-02-14T17:45:40.550907+0000","last_became_active":"2024-02-14T17:45:00.680609+0000","last_became_peered":"2024-02-14T17:45:00.680609+0000","last_unstale":"2024-02-14T17:45:40.550907+0000","last_undegraded":"2024-02-14T17:45:40.550907+0000","last_fullsized":"2024-02-14T17:45:40.550907+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:50.502365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228069+0000","last_change":"2024-02-14T17:45:04.715751+0000","last_active":"2024-02-14T17:45:40.228069+0000","last_peered":"2024-02-14T17:45:40.228069+0000","last_clean":"2024-02-14T17:45:40.228069+0000","last_became_active":"2024-02-14T17:45:04.715434+0000","last_became_peered":"2024-02-14T17:45:04.715434+0000","last_unstale":"2024-02-14T17:45:40.228069+0000","last_undegraded":"2024-02-14T17:45:40.228069+0000","last_fullsized":"2024-02-14T17:45:40.228069+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:25.602332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550359+0000","last_change":"2024-02-14T17:45:02.698488+0000","last_active":"2024-02-14T17:45:40.550359+0000","last_peered":"2024-02-14T17:45:40.550359+0000","last_clean":"2024-02-14T17:45:40.550359+0000","last_became_active":"2024-02-14T17:45:02.698214+0000","last_became_peered":"2024-02-14T17:45:02.698214+0000","last_unstale":"2024-02-14T17:45:40.550359+0000","last_undegraded":"2024-02-14T17:45:40.550359+0000","last_fullsized":"2024-02-14T17:45:40.550359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:43:03.974320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551407+0000","last_change":"2024-02-14T17:44:58.676165+0000","last_active":"2024-02-14T17:45:40.551407+0000","last_peered":"2024-02-14T17:45:40.551407+0000","last_clean":"2024-02-14T17:45:40.551407+0000","last_became_active":"2024-02-14T17:44:58.675840+0000","last_became_peered":"2024-02-14T17:44:58.675840+0000","last_unstale":"2024-02-14T17:45:40.551407+0000","last_undegraded":"2024-02-14T17:45:40.551407+0000","last_fullsized":"2024-02-14T17:45:40.551407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:22:51.275457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":35,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551023+0000","last_change":"2024-02-14T17:45:00.691681+0000","last_active":"2024-02-14T17:45:40.551023+0000","last_peered":"2024-02-14T17:45:40.551023+0000","last_clean":"2024-02-14T17:45:40.551023+0000","last_became_active":"2024-02-14T17:45:00.691363+0000","last_became_peered":"2024-02-14T17:45:00.691363+0000","last_unstale":"2024-02-14T17:45:40.551023+0000","last_undegraded":"2024-02-14T17:45:40.551023+0000","last_fullsized":"2024-02-14T17:45:40.551023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:15:43.542448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551591+0000","last_change":"2024-02-14T17:45:04.949068+0000","last_active":"2024-02-14T17:45:40.551591+0000","last_peered":"2024-02-14T17:45:40.551591+0000","last_clean":"2024-02-14T17:45:40.551591+0000","last_became_active":"2024-02-14T17:45:04.948105+0000","last_became_peered":"2024-02-14T17:45:04.948105+0000","last_unstale":"2024-02-14T17:45:40.551591+0000","last_undegraded":"2024-02-14T17:45:40.551591+0000","last_fullsized":"2024-02-14T17:45:40.551591+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:03:52.607873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228314+0000","last_change":"2024-02-14T17:45:02.703407+0000","last_active":"2024-02-14T17:45:40.228314+0000","last_peered":"2024-02-14T17:45:40.228314+0000","last_clean":"2024-02-14T17:45:40.228314+0000","last_became_active":"2024-02-14T17:45:02.703138+0000","last_became_peered":"2024-02-14T17:45:02.703138+0000","last_unstale":"2024-02-14T17:45:40.228314+0000","last_undegraded":"2024-02-14T17:45:40.228314+0000","last_fullsized":"2024-02-14T17:45:40.228314+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:44:59.159667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332854+0000","last_change":"2024-02-14T17:44:58.670283+0000","last_active":"2024-02-14T17:45:40.332854+0000","last_peered":"2024-02-14T17:45:40.332854+0000","last_clean":"2024-02-14T17:45:40.332854+0000","last_became_active":"2024-02-14T17:44:58.669789+0000","last_became_peered":"2024-02-14T17:44:58.669789+0000","last_unstale":"2024-02-14T17:45:40.332854+0000","last_undegraded":"2024-02-14T17:45:40.332854+0000","last_fullsized":"2024-02-14T17:45:40.332854+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:31:59.756475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551180+0000","last_change":"2024-02-14T17:45:00.686193+0000","last_active":"2024-02-14T17:45:40.551180+0000","last_peered":"2024-02-14T17:45:40.551180+0000","last_clean":"2024-02-14T17:45:40.551180+0000","last_became_active":"2024-02-14T17:45:00.685889+0000","last_became_peered":"2024-02-14T17:45:00.685889+0000","last_unstale":"2024-02-14T17:45:40.551180+0000","last_undegraded":"2024-02-14T17:45:40.551180+0000","last_fullsized":"2024-02-14T17:45:40.551180+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:52:33.932121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"68'1","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550376+0000","last_change":"2024-02-14T17:45:04.716983+0000","last_active":"2024-02-14T17:45:40.550376+0000","last_peered":"2024-02-14T17:45:40.550376+0000","last_clean":"2024-02-14T17:45:40.550376+0000","last_became_active":"2024-02-14T17:45:04.716647+0000","last_became_peered":"2024-02-14T17:45:04.716647+0000","last_unstale":"2024-02-14T17:45:40.550376+0000","last_undegraded":"2024-02-14T17:45:40.550376+0000","last_fullsized":"2024-02-14T17:45:40.550376+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:23:25.652847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551876+0000","last_change":"2024-02-14T17:45:02.706533+0000","last_active":"2024-02-14T17:45:40.551876+0000","last_peered":"2024-02-14T17:45:40.551876+0000","last_clean":"2024-02-14T17:45:40.551876+0000","last_became_active":"2024-02-14T17:45:02.706138+0000","last_became_peered":"2024-02-14T17:45:02.706138+0000","last_unstale":"2024-02-14T17:45:40.551876+0000","last_undegraded":"2024-02-14T17:45:40.551876+0000","last_fullsized":"2024-02-14T17:45:40.551876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:23:18.462462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228401+0000","last_change":"2024-02-14T17:44:58.675882+0000","last_active":"2024-02-14T17:45:40.228401+0000","last_peered":"2024-02-14T17:45:40.228401+0000","last_clean":"2024-02-14T17:45:40.228401+0000","last_became_active":"2024-02-14T17:44:58.674106+0000","last_became_peered":"2024-02-14T17:44:58.674106+0000","last_unstale":"2024-02-14T17:45:40.228401+0000","last_undegraded":"2024-02-14T17:45:40.228401+0000","last_fullsized":"2024-02-14T17:45:40.228401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:18:00.721303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223930+0000","last_change":"2024-02-14T17:45:00.680148+0000","last_active":"2024-02-14T17:45:40.223930+0000","last_peered":"2024-02-14T17:45:40.223930+0000","last_clean":"2024-02-14T17:45:40.223930+0000","last_became_active":"2024-02-14T17:45:00.679831+0000","last_became_peered":"2024-02-14T17:45:00.679831+0000","last_unstale":"2024-02-14T17:45:40.223930+0000","last_undegraded":"2024-02-14T17:45:40.223930+0000","last_fullsized":"2024-02-14T17:45:40.223930+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:25:59.283994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550816+0000","last_change":"2024-02-14T17:45:04.713479+0000","last_active":"2024-02-14T17:45:40.550816+0000","last_peered":"2024-02-14T17:45:40.550816+0000","last_clean":"2024-02-14T17:45:40.550816+0000","last_became_active":"2024-02-14T17:45:04.713154+0000","last_became_peered":"2024-02-14T17:45:04.713154+0000","last_unstale":"2024-02-14T17:45:40.550816+0000","last_undegraded":"2024-02-14T17:45:40.550816+0000","last_fullsized":"2024-02-14T17:45:40.550816+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:42:06.688967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223724+0000","last_change":"2024-02-14T17:45:02.700634+0000","last_active":"2024-02-14T17:45:40.223724+0000","last_peered":"2024-02-14T17:45:40.223724+0000","last_clean":"2024-02-14T17:45:40.223724+0000","last_became_active":"2024-02-14T17:45:02.699623+0000","last_became_peered":"2024-02-14T17:45:02.699623+0000","last_unstale":"2024-02-14T17:45:40.223724+0000","last_undegraded":"2024-02-14T17:45:40.223724+0000","last_fullsized":"2024-02-14T17:45:40.223724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:16:43.321855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223008+0000","last_change":"2024-02-14T17:44:58.674980+0000","last_active":"2024-02-14T17:45:40.223008+0000","last_peered":"2024-02-14T17:45:40.223008+0000","last_clean":"2024-02-14T17:45:40.223008+0000","last_became_active":"2024-02-14T17:44:58.674627+0000","last_became_peered":"2024-02-14T17:44:58.674627+0000","last_unstale":"2024-02-14T17:45:40.223008+0000","last_undegraded":"2024-02-14T17:45:40.223008+0000","last_fullsized":"2024-02-14T17:45:40.223008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:00:56.671226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222860+0000","last_change":"2024-02-14T17:45:00.678716+0000","last_active":"2024-02-14T17:45:40.222860+0000","last_peered":"2024-02-14T17:45:40.222860+0000","last_clean":"2024-02-14T17:45:40.222860+0000","last_became_active":"2024-02-14T17:45:00.678387+0000","last_became_peered":"2024-02-14T17:45:00.678387+0000","last_unstale":"2024-02-14T17:45:40.222860+0000","last_undegraded":"2024-02-14T17:45:40.222860+0000","last_fullsized":"2024-02-14T17:45:40.222860+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:28:27.757181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550863+0000","last_change":"2024-02-14T17:45:04.716655+0000","last_active":"2024-02-14T17:45:40.550863+0000","last_peered":"2024-02-14T17:45:40.550863+0000","last_clean":"2024-02-14T17:45:40.550863+0000","last_became_active":"2024-02-14T17:45:04.716340+0000","last_became_peered":"2024-02-14T17:45:04.716340+0000","last_unstale":"2024-02-14T17:45:40.550863+0000","last_undegraded":"2024-02-14T17:45:40.550863+0000","last_fullsized":"2024-02-14T17:45:40.550863+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:11:25.779867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:08.731839+0000","last_change":"2024-02-14T17:45:02.703704+0000","last_active":"2024-02-14T17:46:08.731839+0000","last_peered":"2024-02-14T17:46:08.731839+0000","last_clean":"2024-02-14T17:46:08.731839+0000","last_became_active":"2024-02-14T17:45:02.703440+0000","last_became_peered":"2024-02-14T17:45:02.703440+0000","last_unstale":"2024-02-14T17:46:08.731839+0000","last_undegraded":"2024-02-14T17:46:08.731839+0000","last_fullsized":"2024-02-14T17:46:08.731839+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:47:11.876537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228197+0000","last_change":"2024-02-14T17:44:58.676173+0000","last_active":"2024-02-14T17:45:40.228197+0000","last_peered":"2024-02-14T17:45:40.228197+0000","last_clean":"2024-02-14T17:45:40.228197+0000","last_became_active":"2024-02-14T17:44:58.674805+0000","last_became_peered":"2024-02-14T17:44:58.674805+0000","last_unstale":"2024-02-14T17:45:40.228197+0000","last_undegraded":"2024-02-14T17:45:40.228197+0000","last_fullsized":"2024-02-14T17:45:40.228197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:32:18.332190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552145+0000","last_change":"2024-02-14T17:45:00.691862+0000","last_active":"2024-02-14T17:45:40.552145+0000","last_peered":"2024-02-14T17:45:40.552145+0000","last_clean":"2024-02-14T17:45:40.552145+0000","last_became_active":"2024-02-14T17:45:00.691496+0000","last_became_peered":"2024-02-14T17:45:00.691496+0000","last_unstale":"2024-02-14T17:45:40.552145+0000","last_undegraded":"2024-02-14T17:45:40.552145+0000","last_fullsized":"2024-02-14T17:45:40.552145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:47:25.258058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223380+0000","last_change":"2024-02-14T17:45:04.715930+0000","last_active":"2024-02-14T17:45:40.223380+0000","last_peered":"2024-02-14T17:45:40.223380+0000","last_clean":"2024-02-14T17:45:40.223380+0000","last_became_active":"2024-02-14T17:45:04.715620+0000","last_became_peered":"2024-02-14T17:45:04.715620+0000","last_unstale":"2024-02-14T17:45:40.223380+0000","last_undegraded":"2024-02-14T17:45:40.223380+0000","last_fullsized":"2024-02-14T17:45:40.223380+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:36:11.365534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:08.731798+0000","last_change":"2024-02-14T17:45:02.702549+0000","last_active":"2024-02-14T17:46:08.731798+0000","last_peered":"2024-02-14T17:46:08.731798+0000","last_clean":"2024-02-14T17:46:08.731798+0000","last_became_active":"2024-02-14T17:45:02.702245+0000","last_became_peered":"2024-02-14T17:45:02.702245+0000","last_unstale":"2024-02-14T17:46:08.731798+0000","last_undegraded":"2024-02-14T17:46:08.731798+0000","last_fullsized":"2024-02-14T17:46:08.731798+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:51:40.686789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550638+0000","last_change":"2024-02-14T17:44:58.678766+0000","last_active":"2024-02-14T17:45:40.550638+0000","last_peered":"2024-02-14T17:45:40.550638+0000","last_clean":"2024-02-14T17:45:40.550638+0000","last_became_active":"2024-02-14T17:44:58.678297+0000","last_became_peered":"2024-02-14T17:44:58.678297+0000","last_unstale":"2024-02-14T17:45:40.550638+0000","last_undegraded":"2024-02-14T17:45:40.550638+0000","last_fullsized":"2024-02-14T17:45:40.550638+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:38:13.004885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332940+0000","last_change":"2024-02-14T17:45:00.687236+0000","last_active":"2024-02-14T17:45:40.332940+0000","last_peered":"2024-02-14T17:45:40.332940+0000","last_clean":"2024-02-14T17:45:40.332940+0000","last_became_active":"2024-02-14T17:45:00.686891+0000","last_became_peered":"2024-02-14T17:45:00.686891+0000","last_unstale":"2024-02-14T17:45:40.332940+0000","last_undegraded":"2024-02-14T17:45:40.332940+0000","last_fullsized":"2024-02-14T17:45:40.332940+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:03:47.323021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222545+0000","last_change":"2024-02-14T17:45:04.709014+0000","last_active":"2024-02-14T17:45:40.222545+0000","last_peered":"2024-02-14T17:45:40.222545+0000","last_clean":"2024-02-14T17:45:40.222545+0000","last_became_active":"2024-02-14T17:45:04.708665+0000","last_became_peered":"2024-02-14T17:45:04.708665+0000","last_unstale":"2024-02-14T17:45:40.222545+0000","last_undegraded":"2024-02-14T17:45:40.222545+0000","last_fullsized":"2024-02-14T17:45:40.222545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:04:16.381858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552724+0000","last_change":"2024-02-14T17:45:02.702276+0000","last_active":"2024-02-14T17:45:40.552724+0000","last_peered":"2024-02-14T17:45:40.552724+0000","last_clean":"2024-02-14T17:45:40.552724+0000","last_became_active":"2024-02-14T17:45:02.701897+0000","last_became_peered":"2024-02-14T17:45:02.701897+0000","last_unstale":"2024-02-14T17:45:40.552724+0000","last_undegraded":"2024-02-14T17:45:40.552724+0000","last_fullsized":"2024-02-14T17:45:40.552724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:25:42.863762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228101+0000","last_change":"2024-02-14T17:44:58.675276+0000","last_active":"2024-02-14T17:45:40.228101+0000","last_peered":"2024-02-14T17:45:40.228101+0000","last_clean":"2024-02-14T17:45:40.228101+0000","last_became_active":"2024-02-14T17:44:58.673558+0000","last_became_peered":"2024-02-14T17:44:58.673558+0000","last_unstale":"2024-02-14T17:45:40.228101+0000","last_undegraded":"2024-02-14T17:45:40.228101+0000","last_fullsized":"2024-02-14T17:45:40.228101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:26:30.585198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551295+0000","last_change":"2024-02-14T17:45:00.687249+0000","last_active":"2024-02-14T17:45:40.551295+0000","last_peered":"2024-02-14T17:45:40.551295+0000","last_clean":"2024-02-14T17:45:40.551295+0000","last_became_active":"2024-02-14T17:45:00.686946+0000","last_became_peered":"2024-02-14T17:45:00.686946+0000","last_unstale":"2024-02-14T17:45:40.551295+0000","last_undegraded":"2024-02-14T17:45:40.551295+0000","last_fullsized":"2024-02-14T17:45:40.551295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:50:00.158912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228553+0000","last_change":"2024-02-14T17:45:04.949232+0000","last_active":"2024-02-14T17:45:40.228553+0000","last_peered":"2024-02-14T17:45:40.228553+0000","last_clean":"2024-02-14T17:45:40.228553+0000","last_became_active":"2024-02-14T17:45:04.948928+0000","last_became_peered":"2024-02-14T17:45:04.948928+0000","last_unstale":"2024-02-14T17:45:40.228553+0000","last_undegraded":"2024-02-14T17:45:40.228553+0000","last_fullsized":"2024-02-14T17:45:40.228553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:34:05.054497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332914+0000","last_change":"2024-02-14T17:45:02.698653+0000","last_active":"2024-02-14T17:45:40.332914+0000","last_peered":"2024-02-14T17:45:40.332914+0000","last_clean":"2024-02-14T17:45:40.332914+0000","last_became_active":"2024-02-14T17:45:02.698359+0000","last_became_peered":"2024-02-14T17:45:02.698359+0000","last_unstale":"2024-02-14T17:45:40.332914+0000","last_undegraded":"2024-02-14T17:45:40.332914+0000","last_fullsized":"2024-02-14T17:45:40.332914+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:36:56.029599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550299+0000","last_change":"2024-02-14T17:44:58.674466+0000","last_active":"2024-02-14T17:45:40.550299+0000","last_peered":"2024-02-14T17:45:40.550299+0000","last_clean":"2024-02-14T17:45:40.550299+0000","last_became_active":"2024-02-14T17:44:58.674159+0000","last_became_peered":"2024-02-14T17:44:58.674159+0000","last_unstale":"2024-02-14T17:45:40.550299+0000","last_undegraded":"2024-02-14T17:45:40.550299+0000","last_fullsized":"2024-02-14T17:45:40.550299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:49:04.214383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":38,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552815+0000","last_change":"2024-02-14T17:45:00.692787+0000","last_active":"2024-02-14T17:45:40.552815+0000","last_peered":"2024-02-14T17:45:40.552815+0000","last_clean":"2024-02-14T17:45:40.552815+0000","last_became_active":"2024-02-14T17:45:00.692435+0000","last_became_peered":"2024-02-14T17:45:00.692435+0000","last_unstale":"2024-02-14T17:45:40.552815+0000","last_undegraded":"2024-02-14T17:45:40.552815+0000","last_fullsized":"2024-02-14T17:45:40.552815+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:34:06.591968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332585+0000","last_change":"2024-02-14T17:45:04.948589+0000","last_active":"2024-02-14T17:45:40.332585+0000","last_peered":"2024-02-14T17:45:40.332585+0000","last_clean":"2024-02-14T17:45:40.332585+0000","last_became_active":"2024-02-14T17:45:04.948300+0000","last_became_peered":"2024-02-14T17:45:04.948300+0000","last_unstale":"2024-02-14T17:45:40.332585+0000","last_undegraded":"2024-02-14T17:45:40.332585+0000","last_fullsized":"2024-02-14T17:45:40.332585+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:20:16.805135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"61'2","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228721+0000","last_change":"2024-02-14T17:44:58.674641+0000","last_active":"2024-02-14T17:45:40.228721+0000","last_peered":"2024-02-14T17:45:40.228721+0000","last_clean":"2024-02-14T17:45:40.228721+0000","last_became_active":"2024-02-14T17:44:58.674354+0000","last_became_peered":"2024-02-14T17:44:58.674354+0000","last_unstale":"2024-02-14T17:45:40.228721+0000","last_undegraded":"2024-02-14T17:45:40.228721+0000","last_fullsized":"2024-02-14T17:45:40.228721+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:37:19.147594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332345+0000","last_change":"2024-02-14T17:45:00.684090+0000","last_active":"2024-02-14T17:45:40.332345+0000","last_peered":"2024-02-14T17:45:40.332345+0000","last_clean":"2024-02-14T17:45:40.332345+0000","last_became_active":"2024-02-14T17:45:00.683741+0000","last_became_peered":"2024-02-14T17:45:00.683741+0000","last_unstale":"2024-02-14T17:45:40.332345+0000","last_undegraded":"2024-02-14T17:45:40.332345+0000","last_fullsized":"2024-02-14T17:45:40.332345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:29:53.573374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:08.731902+0000","last_change":"2024-02-14T17:45:02.704798+0000","last_active":"2024-02-14T17:46:08.731902+0000","last_peered":"2024-02-14T17:46:08.731902+0000","last_clean":"2024-02-14T17:46:08.731902+0000","last_became_active":"2024-02-14T17:45:02.704513+0000","last_became_peered":"2024-02-14T17:45:02.704513+0000","last_unstale":"2024-02-14T17:46:08.731902+0000","last_undegraded":"2024-02-14T17:46:08.731902+0000","last_fullsized":"2024-02-14T17:46:08.731902+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:26:06.860705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551694+0000","last_change":"2024-02-14T17:45:04.711471+0000","last_active":"2024-02-14T17:45:40.551694+0000","last_peered":"2024-02-14T17:45:40.551694+0000","last_clean":"2024-02-14T17:45:40.551694+0000","last_became_active":"2024-02-14T17:45:04.710003+0000","last_became_peered":"2024-02-14T17:45:04.710003+0000","last_unstale":"2024-02-14T17:45:40.551694+0000","last_undegraded":"2024-02-14T17:45:40.551694+0000","last_fullsized":"2024-02-14T17:45:40.551694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:15:06.959381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228544+0000","last_change":"2024-02-14T17:44:58.674669+0000","last_active":"2024-02-14T17:45:40.228544+0000","last_peered":"2024-02-14T17:45:40.228544+0000","last_clean":"2024-02-14T17:45:40.228544+0000","last_became_active":"2024-02-14T17:44:58.673811+0000","last_became_peered":"2024-02-14T17:44:58.673811+0000","last_unstale":"2024-02-14T17:45:40.228544+0000","last_undegraded":"2024-02-14T17:45:40.228544+0000","last_fullsized":"2024-02-14T17:45:40.228544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:46:09.956556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222674+0000","last_change":"2024-02-14T17:45:00.689583+0000","last_active":"2024-02-14T17:45:40.222674+0000","last_peered":"2024-02-14T17:45:40.222674+0000","last_clean":"2024-02-14T17:45:40.222674+0000","last_became_active":"2024-02-14T17:45:00.689263+0000","last_became_peered":"2024-02-14T17:45:00.689263+0000","last_unstale":"2024-02-14T17:45:40.222674+0000","last_undegraded":"2024-02-14T17:45:40.222674+0000","last_fullsized":"2024-02-14T17:45:40.222674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:13.927489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.221907+0000","last_change":"2024-02-14T17:45:02.696880+0000","last_active":"2024-02-14T17:45:40.221907+0000","last_peered":"2024-02-14T17:45:40.221907+0000","last_clean":"2024-02-14T17:45:40.221907+0000","last_became_active":"2024-02-14T17:45:02.696571+0000","last_became_peered":"2024-02-14T17:45:02.696571+0000","last_unstale":"2024-02-14T17:45:40.221907+0000","last_undegraded":"2024-02-14T17:45:40.221907+0000","last_fullsized":"2024-02-14T17:45:40.221907+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:34:20.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.227951+0000","last_change":"2024-02-14T17:45:04.716396+0000","last_active":"2024-02-14T17:45:40.227951+0000","last_peered":"2024-02-14T17:45:40.227951+0000","last_clean":"2024-02-14T17:45:40.227951+0000","last_became_active":"2024-02-14T17:45:04.716095+0000","last_became_peered":"2024-02-14T17:45:04.716095+0000","last_unstale":"2024-02-14T17:45:40.227951+0000","last_undegraded":"2024-02-14T17:45:40.227951+0000","last_fullsized":"2024-02-14T17:45:40.227951+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:27:17.007743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222433+0000","last_change":"2024-02-14T17:45:02.698163+0000","last_active":"2024-02-14T17:45:40.222433+0000","last_peered":"2024-02-14T17:45:40.222433+0000","last_clean":"2024-02-14T17:45:40.222433+0000","last_became_active":"2024-02-14T17:45:02.697830+0000","last_became_peered":"2024-02-14T17:45:02.697830+0000","last_unstale":"2024-02-14T17:45:40.222433+0000","last_undegraded":"2024-02-14T17:45:40.222433+0000","last_fullsized":"2024-02-14T17:45:40.222433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:44:15.039602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228691+0000","last_change":"2024-02-14T17:44:58.672316+0000","last_active":"2024-02-14T17:45:40.228691+0000","last_peered":"2024-02-14T17:45:40.228691+0000","last_clean":"2024-02-14T17:45:40.228691+0000","last_became_active":"2024-02-14T17:44:58.672034+0000","last_became_peered":"2024-02-14T17:44:58.672034+0000","last_unstale":"2024-02-14T17:45:40.228691+0000","last_undegraded":"2024-02-14T17:45:40.228691+0000","last_fullsized":"2024-02-14T17:45:40.228691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:36:46.085665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":86,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550598+0000","last_change":"2024-02-14T17:45:19.858239+0000","last_active":"2024-02-14T17:45:40.550598+0000","last_peered":"2024-02-14T17:45:40.550598+0000","last_clean":"2024-02-14T17:45:40.550598+0000","last_became_active":"2024-02-14T17:45:19.846074+0000","last_became_peered":"2024-02-14T17:45:19.846074+0000","last_unstale":"2024-02-14T17:45:40.550598+0000","last_undegraded":"2024-02-14T17:45:40.550598+0000","last_fullsized":"2024-02-14T17:45:40.550598+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:52:36.982636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550649+0000","last_change":"2024-02-14T17:45:04.716621+0000","last_active":"2024-02-14T17:45:40.550649+0000","last_peered":"2024-02-14T17:45:40.550649+0000","last_clean":"2024-02-14T17:45:40.550649+0000","last_became_active":"2024-02-14T17:45:04.716264+0000","last_became_peered":"2024-02-14T17:45:04.716264+0000","last_unstale":"2024-02-14T17:45:40.550649+0000","last_undegraded":"2024-02-14T17:45:40.550649+0000","last_fullsized":"2024-02-14T17:45:40.550649+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:53:58.392116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222157+0000","last_change":"2024-02-14T17:45:02.699280+0000","last_active":"2024-02-14T17:45:40.222157+0000","last_peered":"2024-02-14T17:45:40.222157+0000","last_clean":"2024-02-14T17:45:40.222157+0000","last_became_active":"2024-02-14T17:45:02.698918+0000","last_became_peered":"2024-02-14T17:45:02.698918+0000","last_unstale":"2024-02-14T17:45:40.222157+0000","last_undegraded":"2024-02-14T17:45:40.222157+0000","last_fullsized":"2024-02-14T17:45:40.222157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:01:12.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332663+0000","last_change":"2024-02-14T17:44:58.670770+0000","last_active":"2024-02-14T17:45:40.332663+0000","last_peered":"2024-02-14T17:45:40.332663+0000","last_clean":"2024-02-14T17:45:40.332663+0000","last_became_active":"2024-02-14T17:44:58.670484+0000","last_became_peered":"2024-02-14T17:44:58.670484+0000","last_unstale":"2024-02-14T17:45:40.332663+0000","last_undegraded":"2024-02-14T17:45:40.332663+0000","last_fullsized":"2024-02-14T17:45:40.332663+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:18:50.908508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551946+0000","last_change":"2024-02-14T17:45:00.679592+0000","last_active":"2024-02-14T17:45:40.551946+0000","last_peered":"2024-02-14T17:45:40.551946+0000","last_clean":"2024-02-14T17:45:40.551946+0000","last_became_active":"2024-02-14T17:45:00.679152+0000","last_became_peered":"2024-02-14T17:45:00.679152+0000","last_unstale":"2024-02-14T17:45:40.551946+0000","last_undegraded":"2024-02-14T17:45:40.551946+0000","last_fullsized":"2024-02-14T17:45:40.551946+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:38:18.492221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552401+0000","last_change":"2024-02-14T17:45:04.709683+0000","last_active":"2024-02-14T17:45:40.552401+0000","last_peered":"2024-02-14T17:45:40.552401+0000","last_clean":"2024-02-14T17:45:40.552401+0000","last_became_active":"2024-02-14T17:45:04.709357+0000","last_became_peered":"2024-02-14T17:45:04.709357+0000","last_unstale":"2024-02-14T17:45:40.552401+0000","last_undegraded":"2024-02-14T17:45:40.552401+0000","last_fullsized":"2024-02-14T17:45:40.552401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:28:17.582701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332425+0000","last_change":"2024-02-14T17:45:01.687587+0000","last_active":"2024-02-14T17:45:40.332425+0000","last_peered":"2024-02-14T17:45:40.332425+0000","last_clean":"2024-02-14T17:45:40.332425+0000","last_became_active":"2024-02-14T17:44:57.663459+0000","last_became_peered":"2024-02-14T17:44:57.663459+0000","last_unstale":"2024-02-14T17:45:40.332425+0000","last_undegraded":"2024-02-14T17:45:40.332425+0000","last_fullsized":"2024-02-14T17:45:40.332425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_clean_scrub_stamp":"2024-02-14T17:44:56.646656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:01:59.063650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054170399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222637+0000","last_change":"2024-02-14T17:44:58.673999+0000","last_active":"2024-02-14T17:45:40.222637+0000","last_peered":"2024-02-14T17:45:40.222637+0000","last_clean":"2024-02-14T17:45:40.222637+0000","last_became_active":"2024-02-14T17:44:58.673669+0000","last_became_peered":"2024-02-14T17:44:58.673669+0000","last_unstale":"2024-02-14T17:45:40.222637+0000","last_undegraded":"2024-02-14T17:45:40.222637+0000","last_fullsized":"2024-02-14T17:45:40.222637+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:33:54.757571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":93,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:08.731577+0000","last_change":"2024-02-14T17:45:00.688975+0000","last_active":"2024-02-14T17:46:08.731577+0000","last_peered":"2024-02-14T17:46:08.731577+0000","last_clean":"2024-02-14T17:46:08.731577+0000","last_became_active":"2024-02-14T17:45:00.688025+0000","last_became_peered":"2024-02-14T17:45:00.688025+0000","last_unstale":"2024-02-14T17:46:08.731577+0000","last_undegraded":"2024-02-14T17:46:08.731577+0000","last_fullsized":"2024-02-14T17:46:08.731577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:10:02.863167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550860+0000","last_change":"2024-02-14T17:45:02.704696+0000","last_active":"2024-02-14T17:45:40.550860+0000","last_peered":"2024-02-14T17:45:40.550860+0000","last_clean":"2024-02-14T17:45:40.550860+0000","last_became_active":"2024-02-14T17:45:02.704381+0000","last_became_peered":"2024-02-14T17:45:02.704381+0000","last_unstale":"2024-02-14T17:45:40.550860+0000","last_undegraded":"2024-02-14T17:45:40.550860+0000","last_fullsized":"2024-02-14T17:45:40.550860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:46:49.394272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228785+0000","last_change":"2024-02-14T17:45:04.948463+0000","last_active":"2024-02-14T17:45:40.228785+0000","last_peered":"2024-02-14T17:45:40.228785+0000","last_clean":"2024-02-14T17:45:40.228785+0000","last_became_active":"2024-02-14T17:45:04.948008+0000","last_became_peered":"2024-02-14T17:45:04.948008+0000","last_unstale":"2024-02-14T17:45:40.228785+0000","last_undegraded":"2024-02-14T17:45:40.228785+0000","last_fullsized":"2024-02-14T17:45:40.228785+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:35:11.920833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223215+0000","last_change":"2024-02-14T17:44:58.673058+0000","last_active":"2024-02-14T17:45:40.223215+0000","last_peered":"2024-02-14T17:45:40.223215+0000","last_clean":"2024-02-14T17:45:40.223215+0000","last_became_active":"2024-02-14T17:44:58.672614+0000","last_became_peered":"2024-02-14T17:44:58.672614+0000","last_unstale":"2024-02-14T17:45:40.223215+0000","last_undegraded":"2024-02-14T17:45:40.223215+0000","last_fullsized":"2024-02-14T17:45:40.223215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:25:10.584142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.221972+0000","last_change":"2024-02-14T17:45:01.685793+0000","last_active":"2024-02-14T17:45:40.221972+0000","last_peered":"2024-02-14T17:45:40.221972+0000","last_clean":"2024-02-14T17:45:40.221972+0000","last_became_active":"2024-02-14T17:44:57.666969+0000","last_became_peered":"2024-02-14T17:44:57.666969+0000","last_unstale":"2024-02-14T17:45:40.221972+0000","last_undegraded":"2024-02-14T17:45:40.221972+0000","last_fullsized":"2024-02-14T17:45:40.221972+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_clean_scrub_stamp":"2024-02-14T17:44:56.646656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:22:40.542091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018462909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222845+0000","last_change":"2024-02-14T17:45:19.850122+0000","last_active":"2024-02-14T17:45:40.222845+0000","last_peered":"2024-02-14T17:45:40.222845+0000","last_clean":"2024-02-14T17:45:40.222845+0000","last_became_active":"2024-02-14T17:45:19.843391+0000","last_became_peered":"2024-02-14T17:45:19.843391+0000","last_unstale":"2024-02-14T17:45:40.222845+0000","last_undegraded":"2024-02-14T17:45:40.222845+0000","last_fullsized":"2024-02-14T17:45:40.222845+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:00.674519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223161+0000","last_change":"2024-02-14T17:45:02.699774+0000","last_active":"2024-02-14T17:45:40.223161+0000","last_peered":"2024-02-14T17:45:40.223161+0000","last_clean":"2024-02-14T17:45:40.223161+0000","last_became_active":"2024-02-14T17:45:02.699466+0000","last_became_peered":"2024-02-14T17:45:02.699466+0000","last_unstale":"2024-02-14T17:45:40.223161+0000","last_undegraded":"2024-02-14T17:45:40.223161+0000","last_fullsized":"2024-02-14T17:45:40.223161+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:42:32.508027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332925+0000","last_change":"2024-02-14T17:45:04.708468+0000","last_active":"2024-02-14T17:45:40.332925+0000","last_peered":"2024-02-14T17:45:40.332925+0000","last_clean":"2024-02-14T17:45:40.332925+0000","last_became_active":"2024-02-14T17:45:04.708183+0000","last_became_peered":"2024-02-14T17:45:04.708183+0000","last_unstale":"2024-02-14T17:45:40.332925+0000","last_undegraded":"2024-02-14T17:45:40.332925+0000","last_fullsized":"2024-02-14T17:45:40.332925+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:36:44.084665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":277,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:42.326798+0000","last_change":"2024-02-14T17:44:47.240826+0000","last_active":"2024-02-14T17:45:42.326798+0000","last_peered":"2024-02-14T17:45:42.326798+0000","last_clean":"2024-02-14T17:45:42.326798+0000","last_became_active":"2024-02-14T17:44:47.240460+0000","last_became_peered":"2024-02-14T17:44:47.240460+0000","last_unstale":"2024-02-14T17:45:42.326798+0000","last_undegraded":"2024-02-14T17:45:42.326798+0000","last_fullsized":"2024-02-14T17:45:42.326798+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:39.018229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:39.018229+0000","last_clean_scrub_stamp":"2024-02-14T17:42:39.018229+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:34:51.986339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551705+0000","last_change":"2024-02-14T17:44:58.671880+0000","last_active":"2024-02-14T17:45:40.551705+0000","last_peered":"2024-02-14T17:45:40.551705+0000","last_clean":"2024-02-14T17:45:40.551705+0000","last_became_active":"2024-02-14T17:44:58.671446+0000","last_became_peered":"2024-02-14T17:44:58.671446+0000","last_unstale":"2024-02-14T17:45:40.551705+0000","last_undegraded":"2024-02-14T17:45:40.551705+0000","last_fullsized":"2024-02-14T17:45:40.551705+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:57.443309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"68'16","reported_seq":67,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:08.732155+0000","last_change":"2024-02-14T17:45:00.685852+0000","last_active":"2024-02-14T17:46:08.732155+0000","last_peered":"2024-02-14T17:46:08.732155+0000","last_clean":"2024-02-14T17:46:08.732155+0000","last_became_active":"2024-02-14T17:45:00.685549+0000","last_became_peered":"2024-02-14T17:45:00.685549+0000","last_unstale":"2024-02-14T17:46:08.732155+0000","last_undegraded":"2024-02-14T17:46:08.732155+0000","last_fullsized":"2024-02-14T17:46:08.732155+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:45:06.359306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228562+0000","last_change":"2024-02-14T17:45:02.705089+0000","last_active":"2024-02-14T17:45:40.228562+0000","last_peered":"2024-02-14T17:45:40.228562+0000","last_clean":"2024-02-14T17:45:40.228562+0000","last_became_active":"2024-02-14T17:45:02.704792+0000","last_became_peered":"2024-02-14T17:45:02.704792+0000","last_unstale":"2024-02-14T17:45:40.228562+0000","last_undegraded":"2024-02-14T17:45:40.228562+0000","last_fullsized":"2024-02-14T17:45:40.228562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:50:05.355736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224072+0000","last_change":"2024-02-14T17:45:04.719067+0000","last_active":"2024-02-14T17:45:40.224072+0000","last_peered":"2024-02-14T17:45:40.224072+0000","last_clean":"2024-02-14T17:45:40.224072+0000","last_became_active":"2024-02-14T17:45:04.718649+0000","last_became_peered":"2024-02-14T17:45:04.718649+0000","last_unstale":"2024-02-14T17:45:40.224072+0000","last_undegraded":"2024-02-14T17:45:40.224072+0000","last_fullsized":"2024-02-14T17:45:40.224072+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:45:48.747278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550943+0000","last_change":"2024-02-14T17:44:58.672992+0000","last_active":"2024-02-14T17:45:40.550943+0000","last_peered":"2024-02-14T17:45:40.550943+0000","last_clean":"2024-02-14T17:45:40.550943+0000","last_became_active":"2024-02-14T17:44:58.672686+0000","last_became_peered":"2024-02-14T17:44:58.672686+0000","last_unstale":"2024-02-14T17:45:40.550943+0000","last_undegraded":"2024-02-14T17:45:40.550943+0000","last_fullsized":"2024-02-14T17:45:40.550943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:11:35.774511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"69'5","reported_seq":115,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:14.617581+0000","last_change":"2024-02-14T17:45:01.690001+0000","last_active":"2024-02-14T17:46:14.617581+0000","last_peered":"2024-02-14T17:46:14.617581+0000","last_clean":"2024-02-14T17:46:14.617581+0000","last_became_active":"2024-02-14T17:44:57.664612+0000","last_became_peered":"2024-02-14T17:44:57.664612+0000","last_unstale":"2024-02-14T17:46:14.617581+0000","last_undegraded":"2024-02-14T17:46:14.617581+0000","last_fullsized":"2024-02-14T17:46:14.617581+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_clean_scrub_stamp":"2024-02-14T17:44:56.646656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:17:03.126359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001530008,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":47,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550956+0000","last_change":"2024-02-14T17:45:00.679791+0000","last_active":"2024-02-14T17:45:40.550956+0000","last_peered":"2024-02-14T17:45:40.550956+0000","last_clean":"2024-02-14T17:45:40.550956+0000","last_became_active":"2024-02-14T17:45:00.679446+0000","last_became_peered":"2024-02-14T17:45:00.679446+0000","last_unstale":"2024-02-14T17:45:40.550956+0000","last_undegraded":"2024-02-14T17:45:40.550956+0000","last_fullsized":"2024-02-14T17:45:40.550956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:54:32.231884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.333074+0000","last_change":"2024-02-14T17:45:02.703677+0000","last_active":"2024-02-14T17:45:40.333074+0000","last_peered":"2024-02-14T17:45:40.333074+0000","last_clean":"2024-02-14T17:45:40.333074+0000","last_became_active":"2024-02-14T17:45:02.703391+0000","last_became_peered":"2024-02-14T17:45:02.703391+0000","last_unstale":"2024-02-14T17:45:40.333074+0000","last_undegraded":"2024-02-14T17:45:40.333074+0000","last_fullsized":"2024-02-14T17:45:40.333074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:15:56.079434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223384+0000","last_change":"2024-02-14T17:45:04.948584+0000","last_active":"2024-02-14T17:45:40.223384+0000","last_peered":"2024-02-14T17:45:40.223384+0000","last_clean":"2024-02-14T17:45:40.223384+0000","last_became_active":"2024-02-14T17:45:04.948214+0000","last_became_peered":"2024-02-14T17:45:04.948214+0000","last_unstale":"2024-02-14T17:45:40.223384+0000","last_undegraded":"2024-02-14T17:45:40.223384+0000","last_fullsized":"2024-02-14T17:45:40.223384+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:27:59.471019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224269+0000","last_change":"2024-02-14T17:44:58.667967+0000","last_active":"2024-02-14T17:45:40.224269+0000","last_peered":"2024-02-14T17:45:40.224269+0000","last_clean":"2024-02-14T17:45:40.224269+0000","last_became_active":"2024-02-14T17:44:58.667678+0000","last_became_peered":"2024-02-14T17:44:58.667678+0000","last_unstale":"2024-02-14T17:45:40.224269+0000","last_undegraded":"2024-02-14T17:45:40.224269+0000","last_fullsized":"2024-02-14T17:45:40.224269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:39:33.937876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":65,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551823+0000","last_change":"2024-02-14T17:45:00.684543+0000","last_active":"2024-02-14T17:45:40.551823+0000","last_peered":"2024-02-14T17:45:40.551823+0000","last_clean":"2024-02-14T17:45:40.551823+0000","last_became_active":"2024-02-14T17:45:00.684197+0000","last_became_peered":"2024-02-14T17:45:00.684197+0000","last_unstale":"2024-02-14T17:45:40.551823+0000","last_undegraded":"2024-02-14T17:45:40.551823+0000","last_fullsized":"2024-02-14T17:45:40.551823+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:20:55.775840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550803+0000","last_change":"2024-02-14T17:45:02.696524+0000","last_active":"2024-02-14T17:45:40.550803+0000","last_peered":"2024-02-14T17:45:40.550803+0000","last_clean":"2024-02-14T17:45:40.550803+0000","last_became_active":"2024-02-14T17:45:02.696246+0000","last_became_peered":"2024-02-14T17:45:02.696246+0000","last_unstale":"2024-02-14T17:45:40.550803+0000","last_undegraded":"2024-02-14T17:45:40.550803+0000","last_fullsized":"2024-02-14T17:45:40.550803+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:31:38.961469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550383+0000","last_change":"2024-02-14T17:45:04.716852+0000","last_active":"2024-02-14T17:45:40.550383+0000","last_peered":"2024-02-14T17:45:40.550383+0000","last_clean":"2024-02-14T17:45:40.550383+0000","last_became_active":"2024-02-14T17:45:04.716543+0000","last_became_peered":"2024-02-14T17:45:04.716543+0000","last_unstale":"2024-02-14T17:45:40.550383+0000","last_undegraded":"2024-02-14T17:45:40.550383+0000","last_fullsized":"2024-02-14T17:45:40.550383+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:03:36.624467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332863+0000","last_change":"2024-02-14T17:44:58.663502+0000","last_active":"2024-02-14T17:45:40.332863+0000","last_peered":"2024-02-14T17:45:40.332863+0000","last_clean":"2024-02-14T17:45:40.332863+0000","last_became_active":"2024-02-14T17:44:58.663180+0000","last_became_peered":"2024-02-14T17:44:58.663180+0000","last_unstale":"2024-02-14T17:45:40.332863+0000","last_undegraded":"2024-02-14T17:45:40.332863+0000","last_fullsized":"2024-02-14T17:45:40.332863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:05:50.227869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223808+0000","last_change":"2024-02-14T17:45:00.681503+0000","last_active":"2024-02-14T17:45:40.223808+0000","last_peered":"2024-02-14T17:45:40.223808+0000","last_clean":"2024-02-14T17:45:40.223808+0000","last_became_active":"2024-02-14T17:45:00.681165+0000","last_became_peered":"2024-02-14T17:45:00.681165+0000","last_unstale":"2024-02-14T17:45:40.223808+0000","last_undegraded":"2024-02-14T17:45:40.223808+0000","last_fullsized":"2024-02-14T17:45:40.223808+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:38:36.501566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:08.731547+0000","last_change":"2024-02-14T17:45:02.698514+0000","last_active":"2024-02-14T17:46:08.731547+0000","last_peered":"2024-02-14T17:46:08.731547+0000","last_clean":"2024-02-14T17:46:08.731547+0000","last_became_active":"2024-02-14T17:45:02.698201+0000","last_became_peered":"2024-02-14T17:45:02.698201+0000","last_unstale":"2024-02-14T17:46:08.731547+0000","last_undegraded":"2024-02-14T17:46:08.731547+0000","last_fullsized":"2024-02-14T17:46:08.731547+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:00:43.539317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228614+0000","last_change":"2024-02-14T17:45:04.949846+0000","last_active":"2024-02-14T17:45:40.228614+0000","last_peered":"2024-02-14T17:45:40.228614+0000","last_clean":"2024-02-14T17:45:40.228614+0000","last_became_active":"2024-02-14T17:45:04.949579+0000","last_became_peered":"2024-02-14T17:45:04.949579+0000","last_unstale":"2024-02-14T17:45:40.228614+0000","last_undegraded":"2024-02-14T17:45:40.228614+0000","last_fullsized":"2024-02-14T17:45:40.228614+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:32:10.549937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"61'1","reported_seq":34,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551086+0000","last_change":"2024-02-14T17:44:58.673750+0000","last_active":"2024-02-14T17:45:40.551086+0000","last_peered":"2024-02-14T17:45:40.551086+0000","last_clean":"2024-02-14T17:45:40.551086+0000","last_became_active":"2024-02-14T17:44:58.673443+0000","last_became_peered":"2024-02-14T17:44:58.673443+0000","last_unstale":"2024-02-14T17:45:40.551086+0000","last_undegraded":"2024-02-14T17:45:40.551086+0000","last_fullsized":"2024-02-14T17:45:40.551086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:39:36.402781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222575+0000","last_change":"2024-02-14T17:45:00.677521+0000","last_active":"2024-02-14T17:45:40.222575+0000","last_peered":"2024-02-14T17:45:40.222575+0000","last_clean":"2024-02-14T17:45:40.222575+0000","last_became_active":"2024-02-14T17:45:00.677104+0000","last_became_peered":"2024-02-14T17:45:00.677104+0000","last_unstale":"2024-02-14T17:45:40.222575+0000","last_undegraded":"2024-02-14T17:45:40.222575+0000","last_fullsized":"2024-02-14T17:45:40.222575+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:46:56.073493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:08.731570+0000","last_change":"2024-02-14T17:45:02.702004+0000","last_active":"2024-02-14T17:46:08.731570+0000","last_peered":"2024-02-14T17:46:08.731570+0000","last_clean":"2024-02-14T17:46:08.731570+0000","last_became_active":"2024-02-14T17:45:02.701645+0000","last_became_peered":"2024-02-14T17:45:02.701645+0000","last_unstale":"2024-02-14T17:46:08.731570+0000","last_undegraded":"2024-02-14T17:46:08.731570+0000","last_fullsized":"2024-02-14T17:46:08.731570+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:18:11.348488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222660+0000","last_change":"2024-02-14T17:45:04.713151+0000","last_active":"2024-02-14T17:45:40.222660+0000","last_peered":"2024-02-14T17:45:40.222660+0000","last_clean":"2024-02-14T17:45:40.222660+0000","last_became_active":"2024-02-14T17:45:04.712794+0000","last_became_peered":"2024-02-14T17:45:04.712794+0000","last_unstale":"2024-02-14T17:45:40.222660+0000","last_undegraded":"2024-02-14T17:45:40.222660+0000","last_fullsized":"2024-02-14T17:45:40.222660+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:57:17.701918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552635+0000","last_change":"2024-02-14T17:44:58.673011+0000","last_active":"2024-02-14T17:45:40.552635+0000","last_peered":"2024-02-14T17:45:40.552635+0000","last_clean":"2024-02-14T17:45:40.552635+0000","last_became_active":"2024-02-14T17:44:58.672321+0000","last_became_peered":"2024-02-14T17:44:58.672321+0000","last_unstale":"2024-02-14T17:45:40.552635+0000","last_undegraded":"2024-02-14T17:45:40.552635+0000","last_fullsized":"2024-02-14T17:45:40.552635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:02:40.531394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":61,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551270+0000","last_change":"2024-02-14T17:45:00.686285+0000","last_active":"2024-02-14T17:45:40.551270+0000","last_peered":"2024-02-14T17:45:40.551270+0000","last_clean":"2024-02-14T17:45:40.551270+0000","last_became_active":"2024-02-14T17:45:00.685970+0000","last_became_peered":"2024-02-14T17:45:00.685970+0000","last_unstale":"2024-02-14T17:45:40.551270+0000","last_undegraded":"2024-02-14T17:45:40.551270+0000","last_fullsized":"2024-02-14T17:45:40.551270+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:24:55.876345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222329+0000","last_change":"2024-02-14T17:45:02.700781+0000","last_active":"2024-02-14T17:45:40.222329+0000","last_peered":"2024-02-14T17:45:40.222329+0000","last_clean":"2024-02-14T17:45:40.222329+0000","last_became_active":"2024-02-14T17:45:02.700443+0000","last_became_peered":"2024-02-14T17:45:02.700443+0000","last_unstale":"2024-02-14T17:45:40.222329+0000","last_undegraded":"2024-02-14T17:45:40.222329+0000","last_fullsized":"2024-02-14T17:45:40.222329+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:59:23.639688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332665+0000","last_change":"2024-02-14T17:45:04.714459+0000","last_active":"2024-02-14T17:45:40.332665+0000","last_peered":"2024-02-14T17:45:40.332665+0000","last_clean":"2024-02-14T17:45:40.332665+0000","last_became_active":"2024-02-14T17:45:04.714147+0000","last_became_peered":"2024-02-14T17:45:04.714147+0000","last_unstale":"2024-02-14T17:45:40.332665+0000","last_undegraded":"2024-02-14T17:45:40.332665+0000","last_fullsized":"2024-02-14T17:45:40.332665+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:00:54.615073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":49,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:08.731861+0000","last_change":"2024-02-14T17:45:02.697200+0000","last_active":"2024-02-14T17:46:08.731861+0000","last_peered":"2024-02-14T17:46:08.731861+0000","last_clean":"2024-02-14T17:46:08.731861+0000","last_became_active":"2024-02-14T17:45:02.696906+0000","last_became_peered":"2024-02-14T17:45:02.696906+0000","last_unstale":"2024-02-14T17:46:08.731861+0000","last_undegraded":"2024-02-14T17:46:08.731861+0000","last_fullsized":"2024-02-14T17:46:08.731861+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:10:19.873730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551176+0000","last_change":"2024-02-14T17:44:58.668323+0000","last_active":"2024-02-14T17:45:40.551176+0000","last_peered":"2024-02-14T17:45:40.551176+0000","last_clean":"2024-02-14T17:45:40.551176+0000","last_became_active":"2024-02-14T17:44:58.667925+0000","last_became_peered":"2024-02-14T17:44:58.667925+0000","last_unstale":"2024-02-14T17:45:40.551176+0000","last_undegraded":"2024-02-14T17:45:40.551176+0000","last_fullsized":"2024-02-14T17:45:40.551176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:31.365397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224858+0000","last_change":"2024-02-14T17:45:00.690181+0000","last_active":"2024-02-14T17:45:40.224858+0000","last_peered":"2024-02-14T17:45:40.224858+0000","last_clean":"2024-02-14T17:45:40.224858+0000","last_became_active":"2024-02-14T17:45:00.689861+0000","last_became_peered":"2024-02-14T17:45:00.689861+0000","last_unstale":"2024-02-14T17:45:40.224858+0000","last_undegraded":"2024-02-14T17:45:40.224858+0000","last_fullsized":"2024-02-14T17:45:40.224858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:37:55.271333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552136+0000","last_change":"2024-02-14T17:45:04.950845+0000","last_active":"2024-02-14T17:45:40.552136+0000","last_peered":"2024-02-14T17:45:40.552136+0000","last_clean":"2024-02-14T17:45:40.552136+0000","last_became_active":"2024-02-14T17:45:04.949811+0000","last_became_peered":"2024-02-14T17:45:04.949811+0000","last_unstale":"2024-02-14T17:45:40.552136+0000","last_undegraded":"2024-02-14T17:45:40.552136+0000","last_fullsized":"2024-02-14T17:45:40.552136+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:17:54.284167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.333070+0000","last_change":"2024-02-14T17:45:02.699552+0000","last_active":"2024-02-14T17:45:40.333070+0000","last_peered":"2024-02-14T17:45:40.333070+0000","last_clean":"2024-02-14T17:45:40.333070+0000","last_became_active":"2024-02-14T17:45:02.699282+0000","last_became_peered":"2024-02-14T17:45:02.699282+0000","last_unstale":"2024-02-14T17:45:40.333070+0000","last_undegraded":"2024-02-14T17:45:40.333070+0000","last_fullsized":"2024-02-14T17:45:40.333070+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:33:29.626427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222365+0000","last_change":"2024-02-14T17:44:58.668707+0000","last_active":"2024-02-14T17:45:40.222365+0000","last_peered":"2024-02-14T17:45:40.222365+0000","last_clean":"2024-02-14T17:45:40.222365+0000","last_became_active":"2024-02-14T17:44:58.668248+0000","last_became_peered":"2024-02-14T17:44:58.668248+0000","last_unstale":"2024-02-14T17:45:40.222365+0000","last_undegraded":"2024-02-14T17:45:40.222365+0000","last_fullsized":"2024-02-14T17:45:40.222365+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:24:28.786641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223299+0000","last_change":"2024-02-14T17:45:00.677662+0000","last_active":"2024-02-14T17:45:40.223299+0000","last_peered":"2024-02-14T17:45:40.223299+0000","last_clean":"2024-02-14T17:45:40.223299+0000","last_became_active":"2024-02-14T17:45:00.677303+0000","last_became_peered":"2024-02-14T17:45:00.677303+0000","last_unstale":"2024-02-14T17:45:40.223299+0000","last_undegraded":"2024-02-14T17:45:40.223299+0000","last_fullsized":"2024-02-14T17:45:40.223299+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:17:30.633703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223015+0000","last_change":"2024-02-14T17:45:04.714817+0000","last_active":"2024-02-14T17:45:40.223015+0000","last_peered":"2024-02-14T17:45:40.223015+0000","last_clean":"2024-02-14T17:45:40.223015+0000","last_became_active":"2024-02-14T17:45:04.714435+0000","last_became_peered":"2024-02-14T17:45:04.714435+0000","last_unstale":"2024-02-14T17:45:40.223015+0000","last_undegraded":"2024-02-14T17:45:40.223015+0000","last_fullsized":"2024-02-14T17:45:40.223015+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:09:21.320734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.225107+0000","last_change":"2024-02-14T17:45:02.702700+0000","last_active":"2024-02-14T17:45:40.225107+0000","last_peered":"2024-02-14T17:45:40.225107+0000","last_clean":"2024-02-14T17:45:40.225107+0000","last_became_active":"2024-02-14T17:45:02.702319+0000","last_became_peered":"2024-02-14T17:45:02.702319+0000","last_unstale":"2024-02-14T17:45:40.225107+0000","last_undegraded":"2024-02-14T17:45:40.225107+0000","last_fullsized":"2024-02-14T17:45:40.225107+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:48:02.062438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550458+0000","last_change":"2024-02-14T17:44:58.672626+0000","last_active":"2024-02-14T17:45:40.550458+0000","last_peered":"2024-02-14T17:45:40.550458+0000","last_clean":"2024-02-14T17:45:40.550458+0000","last_became_active":"2024-02-14T17:44:58.672155+0000","last_became_peered":"2024-02-14T17:44:58.672155+0000","last_unstale":"2024-02-14T17:45:40.550458+0000","last_undegraded":"2024-02-14T17:45:40.550458+0000","last_fullsized":"2024-02-14T17:45:40.550458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:29:55.197436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":57,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223716+0000","last_change":"2024-02-14T17:45:00.679799+0000","last_active":"2024-02-14T17:45:40.223716+0000","last_peered":"2024-02-14T17:45:40.223716+0000","last_clean":"2024-02-14T17:45:40.223716+0000","last_became_active":"2024-02-14T17:45:00.679482+0000","last_became_peered":"2024-02-14T17:45:00.679482+0000","last_unstale":"2024-02-14T17:45:40.223716+0000","last_undegraded":"2024-02-14T17:45:40.223716+0000","last_fullsized":"2024-02-14T17:45:40.223716+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:01:38.324547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223638+0000","last_change":"2024-02-14T17:45:04.713023+0000","last_active":"2024-02-14T17:45:40.223638+0000","last_peered":"2024-02-14T17:45:40.223638+0000","last_clean":"2024-02-14T17:45:40.223638+0000","last_became_active":"2024-02-14T17:45:04.712667+0000","last_became_peered":"2024-02-14T17:45:04.712667+0000","last_unstale":"2024-02-14T17:45:40.223638+0000","last_undegraded":"2024-02-14T17:45:40.223638+0000","last_fullsized":"2024-02-14T17:45:40.223638+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:37:25.104842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:08.731545+0000","last_change":"2024-02-14T17:45:02.700504+0000","last_active":"2024-02-14T17:46:08.731545+0000","last_peered":"2024-02-14T17:46:08.731545+0000","last_clean":"2024-02-14T17:46:08.731545+0000","last_became_active":"2024-02-14T17:45:02.700200+0000","last_became_peered":"2024-02-14T17:45:02.700200+0000","last_unstale":"2024-02-14T17:46:08.731545+0000","last_undegraded":"2024-02-14T17:46:08.731545+0000","last_fullsized":"2024-02-14T17:46:08.731545+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:16.895507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551966+0000","last_change":"2024-02-14T17:44:58.676721+0000","last_active":"2024-02-14T17:45:40.551966+0000","last_peered":"2024-02-14T17:45:40.551966+0000","last_clean":"2024-02-14T17:45:40.551966+0000","last_became_active":"2024-02-14T17:44:58.676400+0000","last_became_peered":"2024-02-14T17:44:58.676400+0000","last_unstale":"2024-02-14T17:45:40.551966+0000","last_undegraded":"2024-02-14T17:45:40.551966+0000","last_fullsized":"2024-02-14T17:45:40.551966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:19:12.259015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222936+0000","last_change":"2024-02-14T17:45:00.688284+0000","last_active":"2024-02-14T17:45:40.222936+0000","last_peered":"2024-02-14T17:45:40.222936+0000","last_clean":"2024-02-14T17:45:40.222936+0000","last_became_active":"2024-02-14T17:45:00.687843+0000","last_became_peered":"2024-02-14T17:45:00.687843+0000","last_unstale":"2024-02-14T17:45:40.222936+0000","last_undegraded":"2024-02-14T17:45:40.222936+0000","last_fullsized":"2024-02-14T17:45:40.222936+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:20.938287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234004,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32548,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712604,"statfs":{"total":95995035648,"available":95961706496,"internally_reserved":0,"allocated":1404928,"data_stored":929851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.59099999999999997}]},{"osd":1,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.52900000000000003}]},{"osd":4,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.65000000000000002}]},{"osd":5,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63200000000000001}]},{"osd":6,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.60499999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495642,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712620,"statfs":{"total":95995035648,"available":95961722880,"internally_reserved":0,"allocated":1384448,"data_stored":928324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.59099999999999997}]},{"osd":1,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.45100000000000001}]},{"osd":2,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53900000000000003}]},{"osd":3,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.623}]},{"osd":4,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.40300000000000002}]},{"osd":5,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.60799999999999998}]},{"osd":7,"last update":"Wed Feb 14 17:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":835584,"data_stored":338806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.33100000000000002,"15min":0.309},"min":{"1min":0.23999999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.34899999999999998,"15min":0.32500000000000001},"min":{"1min":0.24399999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.32200000000000001}]},{"osd":2,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36599999999999999,"15min":0.35299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.40799999999999997,"15min":0.39300000000000002},"min":{"1min":0.27900000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed Feb 14 17:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.46200000000000002},"min":{"1min":0.29699999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.28100000000000003}]},{"osd":4,"last update":"Wed Feb 14 17:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.40300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.60499999999999998}]},{"osd":5,"last update":"Wed Feb 14 17:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58999999999999997,"15min":0.59499999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.65400000000000003}]},{"osd":6,"last update":"Wed Feb 14 17:45:19 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.625},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.52900000000000003}]},{"osd":7,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55800000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1413120,"data_stored":929607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 17:45:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.374,"15min":0.35799999999999998},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.45400000000000001,"15min":0.41999999999999998},"min":{"1min":0.27800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":5.1980000000000004,"5min":5.1980000000000004,"15min":5.1980000000000004},"last":0.53900000000000003}]},{"osd":2,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40500000000000003},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.435,"15min":0.42299999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.8580000000000001,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":0.44}]},{"osd":3,"last update":"Wed Feb 14 17:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.51600000000000001,"15min":0.48499999999999999},"min":{"1min":0.26400000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":9.1219999999999999,"15min":9.1219999999999999},"last":0.501},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.44700000000000001,"15min":0.46300000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.59599999999999997,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.45800000000000002}]},{"osd":4,"last update":"Wed Feb 14 17:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.6459999999999999,"5min":3.6459999999999999,"15min":3.6459999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Wed Feb 14 17:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Wed Feb 14 17:45:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.6030000000000002,"5min":3.6030000000000002,"15min":3.6030000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.60099999999999998}]},{"osd":7,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":4.4130000000000003,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.8959999999999999,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.70799999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":806912,"data_stored":338620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.38500000000000001,"15min":0.38400000000000001},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.52000000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.245},{"interface":"front","average":{"1min":0.373,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51600000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.35199999999999998}]},{"osd":1,"last update":"Wed Feb 14 17:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.24099999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.49099999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42499999999999999,"15min":0.41899999999999998},"min":{"1min":0.26500000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.54000000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.29599999999999999}]},{"osd":3,"last update":"Wed Feb 14 17:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45300000000000001,"15min":0.46200000000000002},"min":{"1min":0.32300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.311},{"interface":"front","average":{"1min":0.437,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.32100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.56399999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.33100000000000002}]},{"osd":4,"last update":"Wed Feb 14 17:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.69299999999999995}]},{"osd":5,"last update":"Wed Feb 14 17:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.57599999999999996}]},{"osd":6,"last update":"Wed Feb 14 17:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.628},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58499999999999996}]},{"osd":7,"last update":"Wed Feb 14 17:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66500000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182440,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":827392,"data_stored":338847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41899999999999998,"15min":0.41599999999999998},"min":{"1min":0.24399999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.25800000000000001}]},{"osd":1,"last update":"Wed Feb 14 17:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.434,"15min":0.442},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.42899999999999999,"15min":0.433},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.316}]},{"osd":2,"last update":"Wed Feb 14 17:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46200000000000002,"15min":0.46000000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.439,"15min":0.439},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.35199999999999998}]},{"osd":4,"last update":"Wed Feb 14 17:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76700000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Feb 14 17:46:01 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Wed Feb 14 17:45:20 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.58999999999999997}]},{"osd":7,"last update":"Wed Feb 14 17:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.746},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.68300000000000005}]}]},{"osd":4,"up_from":30,"seq":128849018914,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":819200,"data_stored":339037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":0.59699999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75800000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65900000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57099999999999995}]},{"osd":1,"last update":"Wed Feb 14 17:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.498},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.56000000000000005},"min":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70599999999999996,"5min":0.87,"15min":0.87},"last":0.65300000000000002}]},{"osd":2,"last update":"Wed Feb 14 17:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68000000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.55300000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66200000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.52700000000000002}]},{"osd":3,"last update":"Wed Feb 14 17:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66800000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.61499999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.54600000000000004}]},{"osd":5,"last update":"Wed Feb 14 17:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.435,"15min":0.435},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.46500000000000002,"15min":0.45700000000000002},"min":{"1min":0.38800000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.67100000000000004}]},{"osd":6,"last update":"Wed Feb 14 17:45:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.622},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.73199999999999998}]},{"osd":7,"last update":"Wed Feb 14 17:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.58099999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757278,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":794624,"data_stored":337896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69099999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.65100000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38200000000000001}]},{"osd":1,"last update":"Wed Feb 14 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68600000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Feb 14 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68700000000000006,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.49099999999999999}]},{"osd":3,"last update":"Wed Feb 14 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71199999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.5},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.55600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.751,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.32800000000000001}]},{"osd":4,"last update":"Wed Feb 14 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36799999999999999,"15min":0.35999999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36699999999999999}]},{"osd":6,"last update":"Wed Feb 14 17:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.311}]},{"osd":7,"last update":"Wed Feb 14 17:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.437},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.44800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T17:46:22.150 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-14T17:46:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:21 smithi179 ceph-mon[33633]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:21 smithi179 ceph-mon[33633]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:46:23.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:22 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:23.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:22 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:22 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:23.656 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:46:24.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:23 smithi100 ceph-mon[30138]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:24.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:23 smithi100 ceph-mon[39557]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:23 smithi179 ceph-mon[33633]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:25.211 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 podman[77133]: 2024-02-14 17:46:25.210222113 +0000 UTC m=+0.231102343 container create f0606bd736060bfb2ce30145098226e4546b772a4561ba3df5f0d7b86dfbb550 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:46:25.799 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 podman[77133]: 2024-02-14 17:46:25.538145225 +0000 UTC m=+0.559025458 container init f0606bd736060bfb2ce30145098226e4546b772a4561ba3df5f0d7b86dfbb550 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:46:25.799 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 podman[77133]: 2024-02-14 17:46:25.543604531 +0000 UTC m=+0.564484764 container start f0606bd736060bfb2ce30145098226e4546b772a4561ba3df5f0d7b86dfbb550 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:46:25.800 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:25.555Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-14T17:46:25.800 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:25.555Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-14T17:46:25.800 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:25.556Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.100 port=9094 2024-02-14T17:46:25.800 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:25.557Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-14T17:46:25.800 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:25.580Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-14T17:46:25.800 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:25.580Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-14T17:46:25.800 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:25.583Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-14T17:46:25.800 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:25.583Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-14T17:46:25.800 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 bash[77133]: f0606bd736060bfb2ce30145098226e4546b772a4561ba3df5f0d7b86dfbb550 2024-02-14T17:46:25.800 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:25 smithi100 systemd[1]: Started Ceph alertmanager.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:46:26.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[30138]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:26.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:26.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.123 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.124 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:46:26.124 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.124 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:46:25] ENGINE Bus STOPPING 2024-02-14T17:46:26.124 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:46:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T17:46:26.124 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:46:25] ENGINE Bus STOPPED 2024-02-14T17:46:26.124 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:46:25] ENGINE Bus STARTING 2024-02-14T17:46:26.124 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:46:25] ENGINE Serving on http://:::9283 2024-02-14T17:46:26.124 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:46:25] ENGINE Bus STARTED 2024-02-14T17:46:26.124 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[39557]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:26.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:26.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:46:26.125 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:25 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.120+0000 7f941535b700 1 -- 172.21.15.100:0/914111631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9410101510 msgr2=0x7f941010d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.120+0000 7f941535b700 1 --2- 172.21.15.100:0/914111631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9410101510 0x7f941010d9b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f940800b0a0 tx=0x7f940800b3b0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.120+0000 7f941535b700 1 -- 172.21.15.100:0/914111631 shutdown_connections 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.120+0000 7f941535b700 1 --2- 172.21.15.100:0/914111631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9410105ab0 0x7f9410105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.120+0000 7f941535b700 1 --2- 172.21.15.100:0/914111631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9410101510 0x7f941010d9b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.120+0000 7f941535b700 1 --2- 172.21.15.100:0/914111631 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9410100b50 0x7f9410100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.120+0000 7f941535b700 1 -- 172.21.15.100:0/914111631 >> 172.21.15.100:0/914111631 conn(0x7f9410077b80 msgr2=0x7f9410077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.120+0000 7f941535b700 1 -- 172.21.15.100:0/914111631 shutdown_connections 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.120+0000 7f941535b700 1 -- 172.21.15.100:0/914111631 wait complete. 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.121+0000 7f941535b700 1 Processor -- start 2024-02-14T17:46:26.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.121+0000 7f941535b700 1 -- start start 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.121+0000 7f941535b700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9410100b50 0x7f9410196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.121+0000 7f941535b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9410101510 0x7f94101965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.121+0000 7f941535b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9410105ab0 0x7f9410196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.121+0000 7f941535b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941019b2e0 con 0x7f9410101510 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.121+0000 7f941535b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941019b450 con 0x7f9410105ab0 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.121+0000 7f941535b700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941019b610 con 0x7f9410100b50 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9410101510 0x7f94101965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940effd700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9410100b50 0x7f9410196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9410101510 0x7f94101965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36198/0 (socket says 172.21.15.100:36198) 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940e7fc700 1 -- 172.21.15.100:0/3620562328 learned_addr learned my addr 172.21.15.100:0/3620562328 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940f7fe700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9410105ab0 0x7f9410196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940e7fc700 1 -- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9410100b50 msgr2=0x7f9410196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940e7fc700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9410100b50 0x7f9410196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940e7fc700 1 -- 172.21.15.100:0/3620562328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9410105ab0 msgr2=0x7f9410196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940e7fc700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9410105ab0 0x7f9410196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940e7fc700 1 -- 172.21.15.100:0/3620562328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9408009d00 con 0x7f9410101510 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f940e7fc700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9410101510 0x7f94101965c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f940400e9a0 tx=0x7f940400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f93f7fff700 1 -- 172.21.15.100:0/3620562328 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404009790 con 0x7f9410101510 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f941535b700 1 -- 172.21.15.100:0/3620562328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941019b8f0 con 0x7f9410101510 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f93f7fff700 1 -- 172.21.15.100:0/3620562328 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f940400ce90 con 0x7f9410101510 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f93f7fff700 1 -- 172.21.15.100:0/3620562328 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404018420 con 0x7f9410101510 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.122+0000 7f941535b700 1 -- 172.21.15.100:0/3620562328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941019be40 con 0x7f9410101510 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.123+0000 7f941535b700 1 -- 172.21.15.100:0/3620562328 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9410068930 con 0x7f9410101510 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.124+0000 7f93f7fff700 1 -- 172.21.15.100:0/3620562328 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f940400ca10 con 0x7f9410101510 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.124+0000 7f93f7fff700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f93f806dac0 0x7f93f806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.124+0000 7f940effd700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f93f806dac0 0x7f93f806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.124+0000 7f93f7fff700 1 -- 172.21.15.100:0/3620562328 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f9404014070 con 0x7f9410101510 2024-02-14T17:46:26.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.125+0000 7f940effd700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f93f806dac0 0x7f93f806ff80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9400007b60 tx=0x7f94000073d0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:26.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.129+0000 7f93f7fff700 1 -- 172.21.15.100:0/3620562328 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9404052d20 con 0x7f9410101510 2024-02-14T17:46:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:25 smithi179 ceph-mon[33633]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:25 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:25 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:25 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:25 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:25 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:25 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:25 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:25 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:46:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:25 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:26.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.321+0000 7f941535b700 1 -- 172.21.15.100:0/3620562328 --> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94101100b0 con 0x7f93f806dac0 2024-02-14T17:46:26.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.331+0000 7f93f7fff700 1 -- 172.21.15.100:0/3620562328 <== mgr.24482 v2:172.21.15.100:6800/1773294710 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+383773 (secure 0 0 0) 0x7f94101100b0 con 0x7f93f806dac0 2024-02-14T17:46:26.338 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:46:26.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 -- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f93f806dac0 msgr2=0x7f93f806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f93f806dac0 0x7f93f806ff80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9400007b60 tx=0x7f94000073d0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 -- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9410101510 msgr2=0x7f94101965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9410101510 0x7f94101965c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f940400e9a0 tx=0x7f940400ecb0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 -- 172.21.15.100:0/3620562328 shutdown_connections 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9410105ab0 0x7f9410196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f93f806dac0 0x7f93f806ff80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f9410100b50 0x7f9410196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 --2- 172.21.15.100:0/3620562328 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9410101510 0x7f94101965c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 -- 172.21.15.100:0/3620562328 >> 172.21.15.100:0/3620562328 conn(0x7f9410077b80 msgr2=0x7f94101046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 -- 172.21.15.100:0/3620562328 shutdown_connections 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:26.335+0000 7f941535b700 1 -- 172.21.15.100:0/3620562328 wait complete. 2024-02-14T17:46:26.342 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-02-14T17:46:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:26 smithi100 ceph-mon[30138]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-14T17:46:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:26 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:46:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:26 smithi100 ceph-mon[30138]: Deploying daemon grafana.a on smithi179 2024-02-14T17:46:27.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:26 smithi100 ceph-mon[39557]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-14T17:46:27.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:26 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:46:27.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:26 smithi100 ceph-mon[39557]: Deploying daemon grafana.a on smithi179 2024-02-14T17:46:27.265 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-02-14T17:46:26.241613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1013,"num_read_kb":859,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":1771654,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":236724,"kb_used_data":8092,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749724492,"statfs":{"total":767960285184,"available":767717879808,"internally_reserved":0,"allocated":8286208,"data_stored":4480988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002671"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222466+0000","last_change":"2024-02-14T17:45:04.714625+0000","last_active":"2024-02-14T17:45:40.222466+0000","last_peered":"2024-02-14T17:45:40.222466+0000","last_clean":"2024-02-14T17:45:40.222466+0000","last_became_active":"2024-02-14T17:45:04.714309+0000","last_became_peered":"2024-02-14T17:45:04.714309+0000","last_unstale":"2024-02-14T17:45:40.222466+0000","last_undegraded":"2024-02-14T17:45:40.222466+0000","last_fullsized":"2024-02-14T17:45:40.222466+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:35:23.162459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223388+0000","last_change":"2024-02-14T17:45:02.703191+0000","last_active":"2024-02-14T17:45:40.223388+0000","last_peered":"2024-02-14T17:45:40.223388+0000","last_clean":"2024-02-14T17:45:40.223388+0000","last_became_active":"2024-02-14T17:45:02.702872+0000","last_became_peered":"2024-02-14T17:45:02.702872+0000","last_unstale":"2024-02-14T17:45:40.223388+0000","last_undegraded":"2024-02-14T17:45:40.223388+0000","last_fullsized":"2024-02-14T17:45:40.223388+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:49:15.006665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551252+0000","last_change":"2024-02-14T17:44:58.678059+0000","last_active":"2024-02-14T17:45:40.551252+0000","last_peered":"2024-02-14T17:45:40.551252+0000","last_clean":"2024-02-14T17:45:40.551252+0000","last_became_active":"2024-02-14T17:44:58.677799+0000","last_became_peered":"2024-02-14T17:44:58.677799+0000","last_unstale":"2024-02-14T17:45:40.551252+0000","last_undegraded":"2024-02-14T17:45:40.551252+0000","last_fullsized":"2024-02-14T17:45:40.551252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:37:25.833721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222312+0000","last_change":"2024-02-14T17:45:00.678177+0000","last_active":"2024-02-14T17:45:40.222312+0000","last_peered":"2024-02-14T17:45:40.222312+0000","last_clean":"2024-02-14T17:45:40.222312+0000","last_became_active":"2024-02-14T17:45:00.677808+0000","last_became_peered":"2024-02-14T17:45:00.677808+0000","last_unstale":"2024-02-14T17:45:40.222312+0000","last_undegraded":"2024-02-14T17:45:40.222312+0000","last_fullsized":"2024-02-14T17:45:40.222312+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:43:40.205055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551788+0000","last_change":"2024-02-14T17:45:04.950149+0000","last_active":"2024-02-14T17:45:40.551788+0000","last_peered":"2024-02-14T17:45:40.551788+0000","last_clean":"2024-02-14T17:45:40.551788+0000","last_became_active":"2024-02-14T17:45:04.949753+0000","last_became_peered":"2024-02-14T17:45:04.949753+0000","last_unstale":"2024-02-14T17:45:40.551788+0000","last_undegraded":"2024-02-14T17:45:40.551788+0000","last_fullsized":"2024-02-14T17:45:40.551788+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:51:31.912410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222319+0000","last_change":"2024-02-14T17:45:02.696717+0000","last_active":"2024-02-14T17:45:40.222319+0000","last_peered":"2024-02-14T17:45:40.222319+0000","last_clean":"2024-02-14T17:45:40.222319+0000","last_became_active":"2024-02-14T17:45:02.696221+0000","last_became_peered":"2024-02-14T17:45:02.696221+0000","last_unstale":"2024-02-14T17:45:40.222319+0000","last_undegraded":"2024-02-14T17:45:40.222319+0000","last_fullsized":"2024-02-14T17:45:40.222319+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:28:48.835710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"61'1","reported_seq":39,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552583+0000","last_change":"2024-02-14T17:44:58.670983+0000","last_active":"2024-02-14T17:45:40.552583+0000","last_peered":"2024-02-14T17:45:40.552583+0000","last_clean":"2024-02-14T17:45:40.552583+0000","last_became_active":"2024-02-14T17:44:58.670683+0000","last_became_peered":"2024-02-14T17:44:58.670683+0000","last_unstale":"2024-02-14T17:45:40.552583+0000","last_undegraded":"2024-02-14T17:45:40.552583+0000","last_fullsized":"2024-02-14T17:45:40.552583+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:47:58.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551320+0000","last_change":"2024-02-14T17:45:00.686822+0000","last_active":"2024-02-14T17:45:40.551320+0000","last_peered":"2024-02-14T17:45:40.551320+0000","last_clean":"2024-02-14T17:45:40.551320+0000","last_became_active":"2024-02-14T17:45:00.686499+0000","last_became_peered":"2024-02-14T17:45:00.686499+0000","last_unstale":"2024-02-14T17:45:40.551320+0000","last_undegraded":"2024-02-14T17:45:40.551320+0000","last_fullsized":"2024-02-14T17:45:40.551320+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:02:08.567402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550597+0000","last_change":"2024-02-14T17:45:04.718285+0000","last_active":"2024-02-14T17:45:40.550597+0000","last_peered":"2024-02-14T17:45:40.550597+0000","last_clean":"2024-02-14T17:45:40.550597+0000","last_became_active":"2024-02-14T17:45:04.717880+0000","last_became_peered":"2024-02-14T17:45:04.717880+0000","last_unstale":"2024-02-14T17:45:40.550597+0000","last_undegraded":"2024-02-14T17:45:40.550597+0000","last_fullsized":"2024-02-14T17:45:40.550597+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:58:29.490294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332545+0000","last_change":"2024-02-14T17:45:02.698551+0000","last_active":"2024-02-14T17:45:40.332545+0000","last_peered":"2024-02-14T17:45:40.332545+0000","last_clean":"2024-02-14T17:45:40.332545+0000","last_became_active":"2024-02-14T17:45:02.698226+0000","last_became_peered":"2024-02-14T17:45:02.698226+0000","last_unstale":"2024-02-14T17:45:40.332545+0000","last_undegraded":"2024-02-14T17:45:40.332545+0000","last_fullsized":"2024-02-14T17:45:40.332545+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:28:24.789299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332744+0000","last_change":"2024-02-14T17:44:58.672885+0000","last_active":"2024-02-14T17:45:40.332744+0000","last_peered":"2024-02-14T17:45:40.332744+0000","last_clean":"2024-02-14T17:45:40.332744+0000","last_became_active":"2024-02-14T17:44:58.672565+0000","last_became_peered":"2024-02-14T17:44:58.672565+0000","last_unstale":"2024-02-14T17:45:40.332744+0000","last_undegraded":"2024-02-14T17:45:40.332744+0000","last_fullsized":"2024-02-14T17:45:40.332744+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:34:41.155588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222830+0000","last_change":"2024-02-14T17:45:00.688304+0000","last_active":"2024-02-14T17:45:40.222830+0000","last_peered":"2024-02-14T17:45:40.222830+0000","last_clean":"2024-02-14T17:45:40.222830+0000","last_became_active":"2024-02-14T17:45:00.687943+0000","last_became_peered":"2024-02-14T17:45:00.687943+0000","last_unstale":"2024-02-14T17:45:40.222830+0000","last_undegraded":"2024-02-14T17:45:40.222830+0000","last_fullsized":"2024-02-14T17:45:40.222830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:50:05.670902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332278+0000","last_change":"2024-02-14T17:45:04.708369+0000","last_active":"2024-02-14T17:45:40.332278+0000","last_peered":"2024-02-14T17:45:40.332278+0000","last_clean":"2024-02-14T17:45:40.332278+0000","last_became_active":"2024-02-14T17:45:04.708036+0000","last_became_peered":"2024-02-14T17:45:04.708036+0000","last_unstale":"2024-02-14T17:45:40.332278+0000","last_undegraded":"2024-02-14T17:45:40.332278+0000","last_fullsized":"2024-02-14T17:45:40.332278+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:19:20.045807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228087+0000","last_change":"2024-02-14T17:45:02.706149+0000","last_active":"2024-02-14T17:45:40.228087+0000","last_peered":"2024-02-14T17:45:40.228087+0000","last_clean":"2024-02-14T17:45:40.228087+0000","last_became_active":"2024-02-14T17:45:02.705784+0000","last_became_peered":"2024-02-14T17:45:02.705784+0000","last_unstale":"2024-02-14T17:45:40.228087+0000","last_undegraded":"2024-02-14T17:45:40.228087+0000","last_fullsized":"2024-02-14T17:45:40.228087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:41:27.847352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332616+0000","last_change":"2024-02-14T17:44:58.674526+0000","last_active":"2024-02-14T17:45:40.332616+0000","last_peered":"2024-02-14T17:45:40.332616+0000","last_clean":"2024-02-14T17:45:40.332616+0000","last_became_active":"2024-02-14T17:44:58.674246+0000","last_became_peered":"2024-02-14T17:44:58.674246+0000","last_unstale":"2024-02-14T17:45:40.332616+0000","last_undegraded":"2024-02-14T17:45:40.332616+0000","last_fullsized":"2024-02-14T17:45:40.332616+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:28:11.075624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"68'5","reported_seq":51,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223064+0000","last_change":"2024-02-14T17:45:19.847889+0000","last_active":"2024-02-14T17:45:40.223064+0000","last_peered":"2024-02-14T17:45:40.223064+0000","last_clean":"2024-02-14T17:45:40.223064+0000","last_became_active":"2024-02-14T17:45:19.843251+0000","last_became_peered":"2024-02-14T17:45:19.843251+0000","last_unstale":"2024-02-14T17:45:40.223064+0000","last_undegraded":"2024-02-14T17:45:40.223064+0000","last_fullsized":"2024-02-14T17:45:40.223064+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:29:56.479700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223436+0000","last_change":"2024-02-14T17:45:04.948373+0000","last_active":"2024-02-14T17:45:40.223436+0000","last_peered":"2024-02-14T17:45:40.223436+0000","last_clean":"2024-02-14T17:45:40.223436+0000","last_became_active":"2024-02-14T17:45:04.948046+0000","last_became_peered":"2024-02-14T17:45:04.948046+0000","last_unstale":"2024-02-14T17:45:40.223436+0000","last_undegraded":"2024-02-14T17:45:40.223436+0000","last_fullsized":"2024-02-14T17:45:40.223436+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:44:07.351791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224063+0000","last_change":"2024-02-14T17:45:02.702711+0000","last_active":"2024-02-14T17:45:40.224063+0000","last_peered":"2024-02-14T17:45:40.224063+0000","last_clean":"2024-02-14T17:45:40.224063+0000","last_became_active":"2024-02-14T17:45:02.702275+0000","last_became_peered":"2024-02-14T17:45:02.702275+0000","last_unstale":"2024-02-14T17:45:40.224063+0000","last_undegraded":"2024-02-14T17:45:40.224063+0000","last_fullsized":"2024-02-14T17:45:40.224063+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:05:52.488550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"61'1","reported_seq":39,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551787+0000","last_change":"2024-02-14T17:44:58.674923+0000","last_active":"2024-02-14T17:45:40.551787+0000","last_peered":"2024-02-14T17:45:40.551787+0000","last_clean":"2024-02-14T17:45:40.551787+0000","last_became_active":"2024-02-14T17:44:58.674588+0000","last_became_peered":"2024-02-14T17:44:58.674588+0000","last_unstale":"2024-02-14T17:45:40.551787+0000","last_undegraded":"2024-02-14T17:45:40.551787+0000","last_fullsized":"2024-02-14T17:45:40.551787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:29:47.449974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222168+0000","last_change":"2024-02-14T17:45:00.686532+0000","last_active":"2024-02-14T17:45:40.222168+0000","last_peered":"2024-02-14T17:45:40.222168+0000","last_clean":"2024-02-14T17:45:40.222168+0000","last_became_active":"2024-02-14T17:45:00.686090+0000","last_became_peered":"2024-02-14T17:45:00.686090+0000","last_unstale":"2024-02-14T17:45:40.222168+0000","last_undegraded":"2024-02-14T17:45:40.222168+0000","last_fullsized":"2024-02-14T17:45:40.222168+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:29:39.498895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552784+0000","last_change":"2024-02-14T17:45:04.950750+0000","last_active":"2024-02-14T17:45:40.552784+0000","last_peered":"2024-02-14T17:45:40.552784+0000","last_clean":"2024-02-14T17:45:40.552784+0000","last_became_active":"2024-02-14T17:45:04.948176+0000","last_became_peered":"2024-02-14T17:45:04.948176+0000","last_unstale":"2024-02-14T17:45:40.552784+0000","last_undegraded":"2024-02-14T17:45:40.552784+0000","last_fullsized":"2024-02-14T17:45:40.552784+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:55:54.998353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222839+0000","last_change":"2024-02-14T17:45:02.699200+0000","last_active":"2024-02-14T17:45:40.222839+0000","last_peered":"2024-02-14T17:45:40.222839+0000","last_clean":"2024-02-14T17:45:40.222839+0000","last_became_active":"2024-02-14T17:45:02.698894+0000","last_became_peered":"2024-02-14T17:45:02.698894+0000","last_unstale":"2024-02-14T17:45:40.222839+0000","last_undegraded":"2024-02-14T17:45:40.222839+0000","last_fullsized":"2024-02-14T17:45:40.222839+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:30:48.740310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222958+0000","last_change":"2024-02-14T17:44:58.671479+0000","last_active":"2024-02-14T17:45:40.222958+0000","last_peered":"2024-02-14T17:45:40.222958+0000","last_clean":"2024-02-14T17:45:40.222958+0000","last_became_active":"2024-02-14T17:44:58.671120+0000","last_became_peered":"2024-02-14T17:44:58.671120+0000","last_unstale":"2024-02-14T17:45:40.222958+0000","last_undegraded":"2024-02-14T17:45:40.222958+0000","last_fullsized":"2024-02-14T17:45:40.222958+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:06:31.844279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"68'12","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552367+0000","last_change":"2024-02-14T17:45:00.685354+0000","last_active":"2024-02-14T17:45:40.552367+0000","last_peered":"2024-02-14T17:45:40.552367+0000","last_clean":"2024-02-14T17:45:40.552367+0000","last_became_active":"2024-02-14T17:45:00.684930+0000","last_became_peered":"2024-02-14T17:45:00.684930+0000","last_unstale":"2024-02-14T17:45:40.552367+0000","last_undegraded":"2024-02-14T17:45:40.552367+0000","last_fullsized":"2024-02-14T17:45:40.552367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:12:13.148305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228387+0000","last_change":"2024-02-14T17:45:04.713180+0000","last_active":"2024-02-14T17:45:40.228387+0000","last_peered":"2024-02-14T17:45:40.228387+0000","last_clean":"2024-02-14T17:45:40.228387+0000","last_became_active":"2024-02-14T17:45:04.712861+0000","last_became_peered":"2024-02-14T17:45:04.712861+0000","last_unstale":"2024-02-14T17:45:40.228387+0000","last_undegraded":"2024-02-14T17:45:40.228387+0000","last_fullsized":"2024-02-14T17:45:40.228387+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:13:47.945433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"68'8","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:18.732549+0000","last_change":"2024-02-14T17:45:02.699995+0000","last_active":"2024-02-14T17:46:18.732549+0000","last_peered":"2024-02-14T17:46:18.732549+0000","last_clean":"2024-02-14T17:46:18.732549+0000","last_became_active":"2024-02-14T17:45:02.699599+0000","last_became_peered":"2024-02-14T17:45:02.699599+0000","last_unstale":"2024-02-14T17:46:18.732549+0000","last_undegraded":"2024-02-14T17:46:18.732549+0000","last_fullsized":"2024-02-14T17:46:18.732549+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:12:23.842221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"61'1","reported_seq":34,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224986+0000","last_change":"2024-02-14T17:44:58.672652+0000","last_active":"2024-02-14T17:45:40.224986+0000","last_peered":"2024-02-14T17:45:40.224986+0000","last_clean":"2024-02-14T17:45:40.224986+0000","last_became_active":"2024-02-14T17:44:58.672328+0000","last_became_peered":"2024-02-14T17:44:58.672328+0000","last_unstale":"2024-02-14T17:45:40.224986+0000","last_undegraded":"2024-02-14T17:45:40.224986+0000","last_fullsized":"2024-02-14T17:45:40.224986+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:41:00.681939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551181+0000","last_change":"2024-02-14T17:45:00.688295+0000","last_active":"2024-02-14T17:45:40.551181+0000","last_peered":"2024-02-14T17:45:40.551181+0000","last_clean":"2024-02-14T17:45:40.551181+0000","last_became_active":"2024-02-14T17:45:00.687983+0000","last_became_peered":"2024-02-14T17:45:00.687983+0000","last_unstale":"2024-02-14T17:45:40.551181+0000","last_undegraded":"2024-02-14T17:45:40.551181+0000","last_fullsized":"2024-02-14T17:45:40.551181+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:30:49.810191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224199+0000","last_change":"2024-02-14T17:45:04.712972+0000","last_active":"2024-02-14T17:45:40.224199+0000","last_peered":"2024-02-14T17:45:40.224199+0000","last_clean":"2024-02-14T17:45:40.224199+0000","last_became_active":"2024-02-14T17:45:04.712646+0000","last_became_peered":"2024-02-14T17:45:04.712646+0000","last_unstale":"2024-02-14T17:45:40.224199+0000","last_undegraded":"2024-02-14T17:45:40.224199+0000","last_fullsized":"2024-02-14T17:45:40.224199+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:30:48.679915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551457+0000","last_change":"2024-02-14T17:45:02.705091+0000","last_active":"2024-02-14T17:45:40.551457+0000","last_peered":"2024-02-14T17:45:40.551457+0000","last_clean":"2024-02-14T17:45:40.551457+0000","last_became_active":"2024-02-14T17:45:02.704276+0000","last_became_peered":"2024-02-14T17:45:02.704276+0000","last_unstale":"2024-02-14T17:45:40.551457+0000","last_undegraded":"2024-02-14T17:45:40.551457+0000","last_fullsized":"2024-02-14T17:45:40.551457+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:09:09.959532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550918+0000","last_change":"2024-02-14T17:44:58.675493+0000","last_active":"2024-02-14T17:45:40.550918+0000","last_peered":"2024-02-14T17:45:40.550918+0000","last_clean":"2024-02-14T17:45:40.550918+0000","last_became_active":"2024-02-14T17:44:58.673635+0000","last_became_peered":"2024-02-14T17:44:58.673635+0000","last_unstale":"2024-02-14T17:45:40.550918+0000","last_undegraded":"2024-02-14T17:45:40.550918+0000","last_fullsized":"2024-02-14T17:45:40.550918+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:52:21.539711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550907+0000","last_change":"2024-02-14T17:45:00.680999+0000","last_active":"2024-02-14T17:45:40.550907+0000","last_peered":"2024-02-14T17:45:40.550907+0000","last_clean":"2024-02-14T17:45:40.550907+0000","last_became_active":"2024-02-14T17:45:00.680609+0000","last_became_peered":"2024-02-14T17:45:00.680609+0000","last_unstale":"2024-02-14T17:45:40.550907+0000","last_undegraded":"2024-02-14T17:45:40.550907+0000","last_fullsized":"2024-02-14T17:45:40.550907+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:37:50.502365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228069+0000","last_change":"2024-02-14T17:45:04.715751+0000","last_active":"2024-02-14T17:45:40.228069+0000","last_peered":"2024-02-14T17:45:40.228069+0000","last_clean":"2024-02-14T17:45:40.228069+0000","last_became_active":"2024-02-14T17:45:04.715434+0000","last_became_peered":"2024-02-14T17:45:04.715434+0000","last_unstale":"2024-02-14T17:45:40.228069+0000","last_undegraded":"2024-02-14T17:45:40.228069+0000","last_fullsized":"2024-02-14T17:45:40.228069+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:36:25.602332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550359+0000","last_change":"2024-02-14T17:45:02.698488+0000","last_active":"2024-02-14T17:45:40.550359+0000","last_peered":"2024-02-14T17:45:40.550359+0000","last_clean":"2024-02-14T17:45:40.550359+0000","last_became_active":"2024-02-14T17:45:02.698214+0000","last_became_peered":"2024-02-14T17:45:02.698214+0000","last_unstale":"2024-02-14T17:45:40.550359+0000","last_undegraded":"2024-02-14T17:45:40.550359+0000","last_fullsized":"2024-02-14T17:45:40.550359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:43:03.974320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551407+0000","last_change":"2024-02-14T17:44:58.676165+0000","last_active":"2024-02-14T17:45:40.551407+0000","last_peered":"2024-02-14T17:45:40.551407+0000","last_clean":"2024-02-14T17:45:40.551407+0000","last_became_active":"2024-02-14T17:44:58.675840+0000","last_became_peered":"2024-02-14T17:44:58.675840+0000","last_unstale":"2024-02-14T17:45:40.551407+0000","last_undegraded":"2024-02-14T17:45:40.551407+0000","last_fullsized":"2024-02-14T17:45:40.551407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:22:51.275457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"68'4","reported_seq":35,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551023+0000","last_change":"2024-02-14T17:45:00.691681+0000","last_active":"2024-02-14T17:45:40.551023+0000","last_peered":"2024-02-14T17:45:40.551023+0000","last_clean":"2024-02-14T17:45:40.551023+0000","last_became_active":"2024-02-14T17:45:00.691363+0000","last_became_peered":"2024-02-14T17:45:00.691363+0000","last_unstale":"2024-02-14T17:45:40.551023+0000","last_undegraded":"2024-02-14T17:45:40.551023+0000","last_fullsized":"2024-02-14T17:45:40.551023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:15:43.542448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551591+0000","last_change":"2024-02-14T17:45:04.949068+0000","last_active":"2024-02-14T17:45:40.551591+0000","last_peered":"2024-02-14T17:45:40.551591+0000","last_clean":"2024-02-14T17:45:40.551591+0000","last_became_active":"2024-02-14T17:45:04.948105+0000","last_became_peered":"2024-02-14T17:45:04.948105+0000","last_unstale":"2024-02-14T17:45:40.551591+0000","last_undegraded":"2024-02-14T17:45:40.551591+0000","last_fullsized":"2024-02-14T17:45:40.551591+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:03:52.607873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228314+0000","last_change":"2024-02-14T17:45:02.703407+0000","last_active":"2024-02-14T17:45:40.228314+0000","last_peered":"2024-02-14T17:45:40.228314+0000","last_clean":"2024-02-14T17:45:40.228314+0000","last_became_active":"2024-02-14T17:45:02.703138+0000","last_became_peered":"2024-02-14T17:45:02.703138+0000","last_unstale":"2024-02-14T17:45:40.228314+0000","last_undegraded":"2024-02-14T17:45:40.228314+0000","last_fullsized":"2024-02-14T17:45:40.228314+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:44:59.159667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332854+0000","last_change":"2024-02-14T17:44:58.670283+0000","last_active":"2024-02-14T17:45:40.332854+0000","last_peered":"2024-02-14T17:45:40.332854+0000","last_clean":"2024-02-14T17:45:40.332854+0000","last_became_active":"2024-02-14T17:44:58.669789+0000","last_became_peered":"2024-02-14T17:44:58.669789+0000","last_unstale":"2024-02-14T17:45:40.332854+0000","last_undegraded":"2024-02-14T17:45:40.332854+0000","last_fullsized":"2024-02-14T17:45:40.332854+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:31:59.756475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551180+0000","last_change":"2024-02-14T17:45:00.686193+0000","last_active":"2024-02-14T17:45:40.551180+0000","last_peered":"2024-02-14T17:45:40.551180+0000","last_clean":"2024-02-14T17:45:40.551180+0000","last_became_active":"2024-02-14T17:45:00.685889+0000","last_became_peered":"2024-02-14T17:45:00.685889+0000","last_unstale":"2024-02-14T17:45:40.551180+0000","last_undegraded":"2024-02-14T17:45:40.551180+0000","last_fullsized":"2024-02-14T17:45:40.551180+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:52:33.932121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"68'1","reported_seq":22,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550376+0000","last_change":"2024-02-14T17:45:04.716983+0000","last_active":"2024-02-14T17:45:40.550376+0000","last_peered":"2024-02-14T17:45:40.550376+0000","last_clean":"2024-02-14T17:45:40.550376+0000","last_became_active":"2024-02-14T17:45:04.716647+0000","last_became_peered":"2024-02-14T17:45:04.716647+0000","last_unstale":"2024-02-14T17:45:40.550376+0000","last_undegraded":"2024-02-14T17:45:40.550376+0000","last_fullsized":"2024-02-14T17:45:40.550376+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:23:25.652847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551876+0000","last_change":"2024-02-14T17:45:02.706533+0000","last_active":"2024-02-14T17:45:40.551876+0000","last_peered":"2024-02-14T17:45:40.551876+0000","last_clean":"2024-02-14T17:45:40.551876+0000","last_became_active":"2024-02-14T17:45:02.706138+0000","last_became_peered":"2024-02-14T17:45:02.706138+0000","last_unstale":"2024-02-14T17:45:40.551876+0000","last_undegraded":"2024-02-14T17:45:40.551876+0000","last_fullsized":"2024-02-14T17:45:40.551876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:23:18.462462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228401+0000","last_change":"2024-02-14T17:44:58.675882+0000","last_active":"2024-02-14T17:45:40.228401+0000","last_peered":"2024-02-14T17:45:40.228401+0000","last_clean":"2024-02-14T17:45:40.228401+0000","last_became_active":"2024-02-14T17:44:58.674106+0000","last_became_peered":"2024-02-14T17:44:58.674106+0000","last_unstale":"2024-02-14T17:45:40.228401+0000","last_undegraded":"2024-02-14T17:45:40.228401+0000","last_fullsized":"2024-02-14T17:45:40.228401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:18:00.721303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223930+0000","last_change":"2024-02-14T17:45:00.680148+0000","last_active":"2024-02-14T17:45:40.223930+0000","last_peered":"2024-02-14T17:45:40.223930+0000","last_clean":"2024-02-14T17:45:40.223930+0000","last_became_active":"2024-02-14T17:45:00.679831+0000","last_became_peered":"2024-02-14T17:45:00.679831+0000","last_unstale":"2024-02-14T17:45:40.223930+0000","last_undegraded":"2024-02-14T17:45:40.223930+0000","last_fullsized":"2024-02-14T17:45:40.223930+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:25:59.283994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550816+0000","last_change":"2024-02-14T17:45:04.713479+0000","last_active":"2024-02-14T17:45:40.550816+0000","last_peered":"2024-02-14T17:45:40.550816+0000","last_clean":"2024-02-14T17:45:40.550816+0000","last_became_active":"2024-02-14T17:45:04.713154+0000","last_became_peered":"2024-02-14T17:45:04.713154+0000","last_unstale":"2024-02-14T17:45:40.550816+0000","last_undegraded":"2024-02-14T17:45:40.550816+0000","last_fullsized":"2024-02-14T17:45:40.550816+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:42:06.688967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223724+0000","last_change":"2024-02-14T17:45:02.700634+0000","last_active":"2024-02-14T17:45:40.223724+0000","last_peered":"2024-02-14T17:45:40.223724+0000","last_clean":"2024-02-14T17:45:40.223724+0000","last_became_active":"2024-02-14T17:45:02.699623+0000","last_became_peered":"2024-02-14T17:45:02.699623+0000","last_unstale":"2024-02-14T17:45:40.223724+0000","last_undegraded":"2024-02-14T17:45:40.223724+0000","last_fullsized":"2024-02-14T17:45:40.223724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:16:43.321855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223008+0000","last_change":"2024-02-14T17:44:58.674980+0000","last_active":"2024-02-14T17:45:40.223008+0000","last_peered":"2024-02-14T17:45:40.223008+0000","last_clean":"2024-02-14T17:45:40.223008+0000","last_became_active":"2024-02-14T17:44:58.674627+0000","last_became_peered":"2024-02-14T17:44:58.674627+0000","last_unstale":"2024-02-14T17:45:40.223008+0000","last_undegraded":"2024-02-14T17:45:40.223008+0000","last_fullsized":"2024-02-14T17:45:40.223008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:00:56.671226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222860+0000","last_change":"2024-02-14T17:45:00.678716+0000","last_active":"2024-02-14T17:45:40.222860+0000","last_peered":"2024-02-14T17:45:40.222860+0000","last_clean":"2024-02-14T17:45:40.222860+0000","last_became_active":"2024-02-14T17:45:00.678387+0000","last_became_peered":"2024-02-14T17:45:00.678387+0000","last_unstale":"2024-02-14T17:45:40.222860+0000","last_undegraded":"2024-02-14T17:45:40.222860+0000","last_fullsized":"2024-02-14T17:45:40.222860+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:28:27.757181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550863+0000","last_change":"2024-02-14T17:45:04.716655+0000","last_active":"2024-02-14T17:45:40.550863+0000","last_peered":"2024-02-14T17:45:40.550863+0000","last_clean":"2024-02-14T17:45:40.550863+0000","last_became_active":"2024-02-14T17:45:04.716340+0000","last_became_peered":"2024-02-14T17:45:04.716340+0000","last_unstale":"2024-02-14T17:45:40.550863+0000","last_undegraded":"2024-02-14T17:45:40.550863+0000","last_fullsized":"2024-02-14T17:45:40.550863+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:11:25.779867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"68'8","reported_seq":51,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:18.732551+0000","last_change":"2024-02-14T17:45:02.703704+0000","last_active":"2024-02-14T17:46:18.732551+0000","last_peered":"2024-02-14T17:46:18.732551+0000","last_clean":"2024-02-14T17:46:18.732551+0000","last_became_active":"2024-02-14T17:45:02.703440+0000","last_became_peered":"2024-02-14T17:45:02.703440+0000","last_unstale":"2024-02-14T17:46:18.732551+0000","last_undegraded":"2024-02-14T17:46:18.732551+0000","last_fullsized":"2024-02-14T17:46:18.732551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:47:11.876537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228197+0000","last_change":"2024-02-14T17:44:58.676173+0000","last_active":"2024-02-14T17:45:40.228197+0000","last_peered":"2024-02-14T17:45:40.228197+0000","last_clean":"2024-02-14T17:45:40.228197+0000","last_became_active":"2024-02-14T17:44:58.674805+0000","last_became_peered":"2024-02-14T17:44:58.674805+0000","last_unstale":"2024-02-14T17:45:40.228197+0000","last_undegraded":"2024-02-14T17:45:40.228197+0000","last_fullsized":"2024-02-14T17:45:40.228197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:32:18.332190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552145+0000","last_change":"2024-02-14T17:45:00.691862+0000","last_active":"2024-02-14T17:45:40.552145+0000","last_peered":"2024-02-14T17:45:40.552145+0000","last_clean":"2024-02-14T17:45:40.552145+0000","last_became_active":"2024-02-14T17:45:00.691496+0000","last_became_peered":"2024-02-14T17:45:00.691496+0000","last_unstale":"2024-02-14T17:45:40.552145+0000","last_undegraded":"2024-02-14T17:45:40.552145+0000","last_fullsized":"2024-02-14T17:45:40.552145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:47:25.258058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223380+0000","last_change":"2024-02-14T17:45:04.715930+0000","last_active":"2024-02-14T17:45:40.223380+0000","last_peered":"2024-02-14T17:45:40.223380+0000","last_clean":"2024-02-14T17:45:40.223380+0000","last_became_active":"2024-02-14T17:45:04.715620+0000","last_became_peered":"2024-02-14T17:45:04.715620+0000","last_unstale":"2024-02-14T17:45:40.223380+0000","last_undegraded":"2024-02-14T17:45:40.223380+0000","last_fullsized":"2024-02-14T17:45:40.223380+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:36:11.365534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"68'8","reported_seq":51,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:18.732692+0000","last_change":"2024-02-14T17:45:02.702549+0000","last_active":"2024-02-14T17:46:18.732692+0000","last_peered":"2024-02-14T17:46:18.732692+0000","last_clean":"2024-02-14T17:46:18.732692+0000","last_became_active":"2024-02-14T17:45:02.702245+0000","last_became_peered":"2024-02-14T17:45:02.702245+0000","last_unstale":"2024-02-14T17:46:18.732692+0000","last_undegraded":"2024-02-14T17:46:18.732692+0000","last_fullsized":"2024-02-14T17:46:18.732692+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:51:40.686789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550638+0000","last_change":"2024-02-14T17:44:58.678766+0000","last_active":"2024-02-14T17:45:40.550638+0000","last_peered":"2024-02-14T17:45:40.550638+0000","last_clean":"2024-02-14T17:45:40.550638+0000","last_became_active":"2024-02-14T17:44:58.678297+0000","last_became_peered":"2024-02-14T17:44:58.678297+0000","last_unstale":"2024-02-14T17:45:40.550638+0000","last_undegraded":"2024-02-14T17:45:40.550638+0000","last_fullsized":"2024-02-14T17:45:40.550638+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:38:13.004885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332940+0000","last_change":"2024-02-14T17:45:00.687236+0000","last_active":"2024-02-14T17:45:40.332940+0000","last_peered":"2024-02-14T17:45:40.332940+0000","last_clean":"2024-02-14T17:45:40.332940+0000","last_became_active":"2024-02-14T17:45:00.686891+0000","last_became_peered":"2024-02-14T17:45:00.686891+0000","last_unstale":"2024-02-14T17:45:40.332940+0000","last_undegraded":"2024-02-14T17:45:40.332940+0000","last_fullsized":"2024-02-14T17:45:40.332940+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:03:47.323021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222545+0000","last_change":"2024-02-14T17:45:04.709014+0000","last_active":"2024-02-14T17:45:40.222545+0000","last_peered":"2024-02-14T17:45:40.222545+0000","last_clean":"2024-02-14T17:45:40.222545+0000","last_became_active":"2024-02-14T17:45:04.708665+0000","last_became_peered":"2024-02-14T17:45:04.708665+0000","last_unstale":"2024-02-14T17:45:40.222545+0000","last_undegraded":"2024-02-14T17:45:40.222545+0000","last_fullsized":"2024-02-14T17:45:40.222545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:04:16.381858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552724+0000","last_change":"2024-02-14T17:45:02.702276+0000","last_active":"2024-02-14T17:45:40.552724+0000","last_peered":"2024-02-14T17:45:40.552724+0000","last_clean":"2024-02-14T17:45:40.552724+0000","last_became_active":"2024-02-14T17:45:02.701897+0000","last_became_peered":"2024-02-14T17:45:02.701897+0000","last_unstale":"2024-02-14T17:45:40.552724+0000","last_undegraded":"2024-02-14T17:45:40.552724+0000","last_fullsized":"2024-02-14T17:45:40.552724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:25:42.863762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228101+0000","last_change":"2024-02-14T17:44:58.675276+0000","last_active":"2024-02-14T17:45:40.228101+0000","last_peered":"2024-02-14T17:45:40.228101+0000","last_clean":"2024-02-14T17:45:40.228101+0000","last_became_active":"2024-02-14T17:44:58.673558+0000","last_became_peered":"2024-02-14T17:44:58.673558+0000","last_unstale":"2024-02-14T17:45:40.228101+0000","last_undegraded":"2024-02-14T17:45:40.228101+0000","last_fullsized":"2024-02-14T17:45:40.228101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:26:30.585198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551295+0000","last_change":"2024-02-14T17:45:00.687249+0000","last_active":"2024-02-14T17:45:40.551295+0000","last_peered":"2024-02-14T17:45:40.551295+0000","last_clean":"2024-02-14T17:45:40.551295+0000","last_became_active":"2024-02-14T17:45:00.686946+0000","last_became_peered":"2024-02-14T17:45:00.686946+0000","last_unstale":"2024-02-14T17:45:40.551295+0000","last_undegraded":"2024-02-14T17:45:40.551295+0000","last_fullsized":"2024-02-14T17:45:40.551295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:50:00.158912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228553+0000","last_change":"2024-02-14T17:45:04.949232+0000","last_active":"2024-02-14T17:45:40.228553+0000","last_peered":"2024-02-14T17:45:40.228553+0000","last_clean":"2024-02-14T17:45:40.228553+0000","last_became_active":"2024-02-14T17:45:04.948928+0000","last_became_peered":"2024-02-14T17:45:04.948928+0000","last_unstale":"2024-02-14T17:45:40.228553+0000","last_undegraded":"2024-02-14T17:45:40.228553+0000","last_fullsized":"2024-02-14T17:45:40.228553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:34:05.054497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332914+0000","last_change":"2024-02-14T17:45:02.698653+0000","last_active":"2024-02-14T17:45:40.332914+0000","last_peered":"2024-02-14T17:45:40.332914+0000","last_clean":"2024-02-14T17:45:40.332914+0000","last_became_active":"2024-02-14T17:45:02.698359+0000","last_became_peered":"2024-02-14T17:45:02.698359+0000","last_unstale":"2024-02-14T17:45:40.332914+0000","last_undegraded":"2024-02-14T17:45:40.332914+0000","last_fullsized":"2024-02-14T17:45:40.332914+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:36:56.029599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550299+0000","last_change":"2024-02-14T17:44:58.674466+0000","last_active":"2024-02-14T17:45:40.550299+0000","last_peered":"2024-02-14T17:45:40.550299+0000","last_clean":"2024-02-14T17:45:40.550299+0000","last_became_active":"2024-02-14T17:44:58.674159+0000","last_became_peered":"2024-02-14T17:44:58.674159+0000","last_unstale":"2024-02-14T17:45:40.550299+0000","last_undegraded":"2024-02-14T17:45:40.550299+0000","last_fullsized":"2024-02-14T17:45:40.550299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:49:04.214383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"68'6","reported_seq":38,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552815+0000","last_change":"2024-02-14T17:45:00.692787+0000","last_active":"2024-02-14T17:45:40.552815+0000","last_peered":"2024-02-14T17:45:40.552815+0000","last_clean":"2024-02-14T17:45:40.552815+0000","last_became_active":"2024-02-14T17:45:00.692435+0000","last_became_peered":"2024-02-14T17:45:00.692435+0000","last_unstale":"2024-02-14T17:45:40.552815+0000","last_undegraded":"2024-02-14T17:45:40.552815+0000","last_fullsized":"2024-02-14T17:45:40.552815+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:34:06.591968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332585+0000","last_change":"2024-02-14T17:45:04.948589+0000","last_active":"2024-02-14T17:45:40.332585+0000","last_peered":"2024-02-14T17:45:40.332585+0000","last_clean":"2024-02-14T17:45:40.332585+0000","last_became_active":"2024-02-14T17:45:04.948300+0000","last_became_peered":"2024-02-14T17:45:04.948300+0000","last_unstale":"2024-02-14T17:45:40.332585+0000","last_undegraded":"2024-02-14T17:45:40.332585+0000","last_fullsized":"2024-02-14T17:45:40.332585+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:20:16.805135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"61'2","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228721+0000","last_change":"2024-02-14T17:44:58.674641+0000","last_active":"2024-02-14T17:45:40.228721+0000","last_peered":"2024-02-14T17:45:40.228721+0000","last_clean":"2024-02-14T17:45:40.228721+0000","last_became_active":"2024-02-14T17:44:58.674354+0000","last_became_peered":"2024-02-14T17:44:58.674354+0000","last_unstale":"2024-02-14T17:45:40.228721+0000","last_undegraded":"2024-02-14T17:45:40.228721+0000","last_fullsized":"2024-02-14T17:45:40.228721+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T21:37:19.147594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332345+0000","last_change":"2024-02-14T17:45:00.684090+0000","last_active":"2024-02-14T17:45:40.332345+0000","last_peered":"2024-02-14T17:45:40.332345+0000","last_clean":"2024-02-14T17:45:40.332345+0000","last_became_active":"2024-02-14T17:45:00.683741+0000","last_became_peered":"2024-02-14T17:45:00.683741+0000","last_unstale":"2024-02-14T17:45:40.332345+0000","last_undegraded":"2024-02-14T17:45:40.332345+0000","last_fullsized":"2024-02-14T17:45:40.332345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:29:53.573374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"68'8","reported_seq":51,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:18.732595+0000","last_change":"2024-02-14T17:45:02.704798+0000","last_active":"2024-02-14T17:46:18.732595+0000","last_peered":"2024-02-14T17:46:18.732595+0000","last_clean":"2024-02-14T17:46:18.732595+0000","last_became_active":"2024-02-14T17:45:02.704513+0000","last_became_peered":"2024-02-14T17:45:02.704513+0000","last_unstale":"2024-02-14T17:46:18.732595+0000","last_undegraded":"2024-02-14T17:46:18.732595+0000","last_fullsized":"2024-02-14T17:46:18.732595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:26:06.860705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551694+0000","last_change":"2024-02-14T17:45:04.711471+0000","last_active":"2024-02-14T17:45:40.551694+0000","last_peered":"2024-02-14T17:45:40.551694+0000","last_clean":"2024-02-14T17:45:40.551694+0000","last_became_active":"2024-02-14T17:45:04.710003+0000","last_became_peered":"2024-02-14T17:45:04.710003+0000","last_unstale":"2024-02-14T17:45:40.551694+0000","last_undegraded":"2024-02-14T17:45:40.551694+0000","last_fullsized":"2024-02-14T17:45:40.551694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:15:06.959381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228544+0000","last_change":"2024-02-14T17:44:58.674669+0000","last_active":"2024-02-14T17:45:40.228544+0000","last_peered":"2024-02-14T17:45:40.228544+0000","last_clean":"2024-02-14T17:45:40.228544+0000","last_became_active":"2024-02-14T17:44:58.673811+0000","last_became_peered":"2024-02-14T17:44:58.673811+0000","last_unstale":"2024-02-14T17:45:40.228544+0000","last_undegraded":"2024-02-14T17:45:40.228544+0000","last_fullsized":"2024-02-14T17:45:40.228544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:46:09.956556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"68'12","reported_seq":52,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222674+0000","last_change":"2024-02-14T17:45:00.689583+0000","last_active":"2024-02-14T17:45:40.222674+0000","last_peered":"2024-02-14T17:45:40.222674+0000","last_clean":"2024-02-14T17:45:40.222674+0000","last_became_active":"2024-02-14T17:45:00.689263+0000","last_became_peered":"2024-02-14T17:45:00.689263+0000","last_unstale":"2024-02-14T17:45:40.222674+0000","last_undegraded":"2024-02-14T17:45:40.222674+0000","last_fullsized":"2024-02-14T17:45:40.222674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:00:13.927489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.221907+0000","last_change":"2024-02-14T17:45:02.696880+0000","last_active":"2024-02-14T17:45:40.221907+0000","last_peered":"2024-02-14T17:45:40.221907+0000","last_clean":"2024-02-14T17:45:40.221907+0000","last_became_active":"2024-02-14T17:45:02.696571+0000","last_became_peered":"2024-02-14T17:45:02.696571+0000","last_unstale":"2024-02-14T17:45:40.221907+0000","last_undegraded":"2024-02-14T17:45:40.221907+0000","last_fullsized":"2024-02-14T17:45:40.221907+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:34:20.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.227951+0000","last_change":"2024-02-14T17:45:04.716396+0000","last_active":"2024-02-14T17:45:40.227951+0000","last_peered":"2024-02-14T17:45:40.227951+0000","last_clean":"2024-02-14T17:45:40.227951+0000","last_became_active":"2024-02-14T17:45:04.716095+0000","last_became_peered":"2024-02-14T17:45:04.716095+0000","last_unstale":"2024-02-14T17:45:40.227951+0000","last_undegraded":"2024-02-14T17:45:40.227951+0000","last_fullsized":"2024-02-14T17:45:40.227951+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:27:17.007743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222433+0000","last_change":"2024-02-14T17:45:02.698163+0000","last_active":"2024-02-14T17:45:40.222433+0000","last_peered":"2024-02-14T17:45:40.222433+0000","last_clean":"2024-02-14T17:45:40.222433+0000","last_became_active":"2024-02-14T17:45:02.697830+0000","last_became_peered":"2024-02-14T17:45:02.697830+0000","last_unstale":"2024-02-14T17:45:40.222433+0000","last_undegraded":"2024-02-14T17:45:40.222433+0000","last_fullsized":"2024-02-14T17:45:40.222433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:44:15.039602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228691+0000","last_change":"2024-02-14T17:44:58.672316+0000","last_active":"2024-02-14T17:45:40.228691+0000","last_peered":"2024-02-14T17:45:40.228691+0000","last_clean":"2024-02-14T17:45:40.228691+0000","last_became_active":"2024-02-14T17:44:58.672034+0000","last_became_peered":"2024-02-14T17:44:58.672034+0000","last_unstale":"2024-02-14T17:45:40.228691+0000","last_undegraded":"2024-02-14T17:45:40.228691+0000","last_fullsized":"2024-02-14T17:45:40.228691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:36:46.085665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"68'19","reported_seq":86,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550598+0000","last_change":"2024-02-14T17:45:19.858239+0000","last_active":"2024-02-14T17:45:40.550598+0000","last_peered":"2024-02-14T17:45:40.550598+0000","last_clean":"2024-02-14T17:45:40.550598+0000","last_became_active":"2024-02-14T17:45:19.846074+0000","last_became_peered":"2024-02-14T17:45:19.846074+0000","last_unstale":"2024-02-14T17:45:40.550598+0000","last_undegraded":"2024-02-14T17:45:40.550598+0000","last_fullsized":"2024-02-14T17:45:40.550598+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:52:36.982636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550649+0000","last_change":"2024-02-14T17:45:04.716621+0000","last_active":"2024-02-14T17:45:40.550649+0000","last_peered":"2024-02-14T17:45:40.550649+0000","last_clean":"2024-02-14T17:45:40.550649+0000","last_became_active":"2024-02-14T17:45:04.716264+0000","last_became_peered":"2024-02-14T17:45:04.716264+0000","last_unstale":"2024-02-14T17:45:40.550649+0000","last_undegraded":"2024-02-14T17:45:40.550649+0000","last_fullsized":"2024-02-14T17:45:40.550649+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:53:58.392116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222157+0000","last_change":"2024-02-14T17:45:02.699280+0000","last_active":"2024-02-14T17:45:40.222157+0000","last_peered":"2024-02-14T17:45:40.222157+0000","last_clean":"2024-02-14T17:45:40.222157+0000","last_became_active":"2024-02-14T17:45:02.698918+0000","last_became_peered":"2024-02-14T17:45:02.698918+0000","last_unstale":"2024-02-14T17:45:40.222157+0000","last_undegraded":"2024-02-14T17:45:40.222157+0000","last_fullsized":"2024-02-14T17:45:40.222157+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:01:12.185695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332663+0000","last_change":"2024-02-14T17:44:58.670770+0000","last_active":"2024-02-14T17:45:40.332663+0000","last_peered":"2024-02-14T17:45:40.332663+0000","last_clean":"2024-02-14T17:45:40.332663+0000","last_became_active":"2024-02-14T17:44:58.670484+0000","last_became_peered":"2024-02-14T17:44:58.670484+0000","last_unstale":"2024-02-14T17:45:40.332663+0000","last_undegraded":"2024-02-14T17:45:40.332663+0000","last_fullsized":"2024-02-14T17:45:40.332663+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:18:50.908508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"68'9","reported_seq":45,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551946+0000","last_change":"2024-02-14T17:45:00.679592+0000","last_active":"2024-02-14T17:45:40.551946+0000","last_peered":"2024-02-14T17:45:40.551946+0000","last_clean":"2024-02-14T17:45:40.551946+0000","last_became_active":"2024-02-14T17:45:00.679152+0000","last_became_peered":"2024-02-14T17:45:00.679152+0000","last_unstale":"2024-02-14T17:45:40.551946+0000","last_undegraded":"2024-02-14T17:45:40.551946+0000","last_fullsized":"2024-02-14T17:45:40.551946+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:38:18.492221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552401+0000","last_change":"2024-02-14T17:45:04.709683+0000","last_active":"2024-02-14T17:45:40.552401+0000","last_peered":"2024-02-14T17:45:40.552401+0000","last_clean":"2024-02-14T17:45:40.552401+0000","last_became_active":"2024-02-14T17:45:04.709357+0000","last_became_peered":"2024-02-14T17:45:04.709357+0000","last_unstale":"2024-02-14T17:45:40.552401+0000","last_undegraded":"2024-02-14T17:45:40.552401+0000","last_fullsized":"2024-02-14T17:45:40.552401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:28:17.582701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332425+0000","last_change":"2024-02-14T17:45:01.687587+0000","last_active":"2024-02-14T17:45:40.332425+0000","last_peered":"2024-02-14T17:45:40.332425+0000","last_clean":"2024-02-14T17:45:40.332425+0000","last_became_active":"2024-02-14T17:44:57.663459+0000","last_became_peered":"2024-02-14T17:44:57.663459+0000","last_unstale":"2024-02-14T17:45:40.332425+0000","last_undegraded":"2024-02-14T17:45:40.332425+0000","last_fullsized":"2024-02-14T17:45:40.332425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_clean_scrub_stamp":"2024-02-14T17:44:56.646656+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:01:59.063650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054170399999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222637+0000","last_change":"2024-02-14T17:44:58.673999+0000","last_active":"2024-02-14T17:45:40.222637+0000","last_peered":"2024-02-14T17:45:40.222637+0000","last_clean":"2024-02-14T17:45:40.222637+0000","last_became_active":"2024-02-14T17:44:58.673669+0000","last_became_peered":"2024-02-14T17:44:58.673669+0000","last_unstale":"2024-02-14T17:45:40.222637+0000","last_undegraded":"2024-02-14T17:45:40.222637+0000","last_fullsized":"2024-02-14T17:45:40.222637+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:33:54.757571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"68'28","reported_seq":95,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:18.732431+0000","last_change":"2024-02-14T17:45:00.688975+0000","last_active":"2024-02-14T17:46:18.732431+0000","last_peered":"2024-02-14T17:46:18.732431+0000","last_clean":"2024-02-14T17:46:18.732431+0000","last_became_active":"2024-02-14T17:45:00.688025+0000","last_became_peered":"2024-02-14T17:45:00.688025+0000","last_unstale":"2024-02-14T17:46:18.732431+0000","last_undegraded":"2024-02-14T17:46:18.732431+0000","last_fullsized":"2024-02-14T17:46:18.732431+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:10:02.863167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550860+0000","last_change":"2024-02-14T17:45:02.704696+0000","last_active":"2024-02-14T17:45:40.550860+0000","last_peered":"2024-02-14T17:45:40.550860+0000","last_clean":"2024-02-14T17:45:40.550860+0000","last_became_active":"2024-02-14T17:45:02.704381+0000","last_became_peered":"2024-02-14T17:45:02.704381+0000","last_unstale":"2024-02-14T17:45:40.550860+0000","last_undegraded":"2024-02-14T17:45:40.550860+0000","last_fullsized":"2024-02-14T17:45:40.550860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:46:49.394272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228785+0000","last_change":"2024-02-14T17:45:04.948463+0000","last_active":"2024-02-14T17:45:40.228785+0000","last_peered":"2024-02-14T17:45:40.228785+0000","last_clean":"2024-02-14T17:45:40.228785+0000","last_became_active":"2024-02-14T17:45:04.948008+0000","last_became_peered":"2024-02-14T17:45:04.948008+0000","last_unstale":"2024-02-14T17:45:40.228785+0000","last_undegraded":"2024-02-14T17:45:40.228785+0000","last_fullsized":"2024-02-14T17:45:40.228785+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:35:11.920833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223215+0000","last_change":"2024-02-14T17:44:58.673058+0000","last_active":"2024-02-14T17:45:40.223215+0000","last_peered":"2024-02-14T17:45:40.223215+0000","last_clean":"2024-02-14T17:45:40.223215+0000","last_became_active":"2024-02-14T17:44:58.672614+0000","last_became_peered":"2024-02-14T17:44:58.672614+0000","last_unstale":"2024-02-14T17:45:40.223215+0000","last_undegraded":"2024-02-14T17:45:40.223215+0000","last_fullsized":"2024-02-14T17:45:40.223215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:25:10.584142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.221972+0000","last_change":"2024-02-14T17:45:01.685793+0000","last_active":"2024-02-14T17:45:40.221972+0000","last_peered":"2024-02-14T17:45:40.221972+0000","last_clean":"2024-02-14T17:45:40.221972+0000","last_became_active":"2024-02-14T17:44:57.666969+0000","last_became_peered":"2024-02-14T17:44:57.666969+0000","last_unstale":"2024-02-14T17:45:40.221972+0000","last_undegraded":"2024-02-14T17:45:40.221972+0000","last_fullsized":"2024-02-14T17:45:40.221972+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_clean_scrub_stamp":"2024-02-14T17:44:56.646656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:22:40.542091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018462909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"68'13","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222845+0000","last_change":"2024-02-14T17:45:19.850122+0000","last_active":"2024-02-14T17:45:40.222845+0000","last_peered":"2024-02-14T17:45:40.222845+0000","last_clean":"2024-02-14T17:45:40.222845+0000","last_became_active":"2024-02-14T17:45:19.843391+0000","last_became_peered":"2024-02-14T17:45:19.843391+0000","last_unstale":"2024-02-14T17:45:40.222845+0000","last_undegraded":"2024-02-14T17:45:40.222845+0000","last_fullsized":"2024-02-14T17:45:40.222845+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:58:00.674519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223161+0000","last_change":"2024-02-14T17:45:02.699774+0000","last_active":"2024-02-14T17:45:40.223161+0000","last_peered":"2024-02-14T17:45:40.223161+0000","last_clean":"2024-02-14T17:45:40.223161+0000","last_became_active":"2024-02-14T17:45:02.699466+0000","last_became_peered":"2024-02-14T17:45:02.699466+0000","last_unstale":"2024-02-14T17:45:40.223161+0000","last_undegraded":"2024-02-14T17:45:40.223161+0000","last_fullsized":"2024-02-14T17:45:40.223161+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:42:32.508027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332925+0000","last_change":"2024-02-14T17:45:04.708468+0000","last_active":"2024-02-14T17:45:40.332925+0000","last_peered":"2024-02-14T17:45:40.332925+0000","last_clean":"2024-02-14T17:45:40.332925+0000","last_became_active":"2024-02-14T17:45:04.708183+0000","last_became_peered":"2024-02-14T17:45:04.708183+0000","last_unstale":"2024-02-14T17:45:40.332925+0000","last_undegraded":"2024-02-14T17:45:40.332925+0000","last_fullsized":"2024-02-14T17:45:40.332925+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:36:44.084665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'128","reported_seq":277,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:42.326798+0000","last_change":"2024-02-14T17:44:47.240826+0000","last_active":"2024-02-14T17:45:42.326798+0000","last_peered":"2024-02-14T17:45:42.326798+0000","last_clean":"2024-02-14T17:45:42.326798+0000","last_became_active":"2024-02-14T17:44:47.240460+0000","last_became_peered":"2024-02-14T17:44:47.240460+0000","last_unstale":"2024-02-14T17:45:42.326798+0000","last_undegraded":"2024-02-14T17:45:42.326798+0000","last_fullsized":"2024-02-14T17:45:42.326798+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:42:39.018229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:42:39.018229+0000","last_clean_scrub_stamp":"2024-02-14T17:42:39.018229+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:34:51.986339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551705+0000","last_change":"2024-02-14T17:44:58.671880+0000","last_active":"2024-02-14T17:45:40.551705+0000","last_peered":"2024-02-14T17:45:40.551705+0000","last_clean":"2024-02-14T17:45:40.551705+0000","last_became_active":"2024-02-14T17:44:58.671446+0000","last_became_peered":"2024-02-14T17:44:58.671446+0000","last_unstale":"2024-02-14T17:45:40.551705+0000","last_undegraded":"2024-02-14T17:45:40.551705+0000","last_fullsized":"2024-02-14T17:45:40.551705+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:17:57.443309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"68'16","reported_seq":69,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:18.732800+0000","last_change":"2024-02-14T17:45:00.685852+0000","last_active":"2024-02-14T17:46:18.732800+0000","last_peered":"2024-02-14T17:46:18.732800+0000","last_clean":"2024-02-14T17:46:18.732800+0000","last_became_active":"2024-02-14T17:45:00.685549+0000","last_became_peered":"2024-02-14T17:45:00.685549+0000","last_unstale":"2024-02-14T17:46:18.732800+0000","last_undegraded":"2024-02-14T17:46:18.732800+0000","last_fullsized":"2024-02-14T17:46:18.732800+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:45:06.359306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228562+0000","last_change":"2024-02-14T17:45:02.705089+0000","last_active":"2024-02-14T17:45:40.228562+0000","last_peered":"2024-02-14T17:45:40.228562+0000","last_clean":"2024-02-14T17:45:40.228562+0000","last_became_active":"2024-02-14T17:45:02.704792+0000","last_became_peered":"2024-02-14T17:45:02.704792+0000","last_unstale":"2024-02-14T17:45:40.228562+0000","last_undegraded":"2024-02-14T17:45:40.228562+0000","last_fullsized":"2024-02-14T17:45:40.228562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:50:05.355736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224072+0000","last_change":"2024-02-14T17:45:04.719067+0000","last_active":"2024-02-14T17:45:40.224072+0000","last_peered":"2024-02-14T17:45:40.224072+0000","last_clean":"2024-02-14T17:45:40.224072+0000","last_became_active":"2024-02-14T17:45:04.718649+0000","last_became_peered":"2024-02-14T17:45:04.718649+0000","last_unstale":"2024-02-14T17:45:40.224072+0000","last_undegraded":"2024-02-14T17:45:40.224072+0000","last_fullsized":"2024-02-14T17:45:40.224072+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:45:48.747278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550943+0000","last_change":"2024-02-14T17:44:58.672992+0000","last_active":"2024-02-14T17:45:40.550943+0000","last_peered":"2024-02-14T17:45:40.550943+0000","last_clean":"2024-02-14T17:45:40.550943+0000","last_became_active":"2024-02-14T17:44:58.672686+0000","last_became_peered":"2024-02-14T17:44:58.672686+0000","last_unstale":"2024-02-14T17:45:40.550943+0000","last_undegraded":"2024-02-14T17:45:40.550943+0000","last_fullsized":"2024-02-14T17:45:40.550943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:11:35.774511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"69'5","reported_seq":125,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:24.632256+0000","last_change":"2024-02-14T17:45:01.690001+0000","last_active":"2024-02-14T17:46:24.632256+0000","last_peered":"2024-02-14T17:46:24.632256+0000","last_clean":"2024-02-14T17:46:24.632256+0000","last_became_active":"2024-02-14T17:44:57.664612+0000","last_became_peered":"2024-02-14T17:44:57.664612+0000","last_unstale":"2024-02-14T17:46:24.632256+0000","last_undegraded":"2024-02-14T17:46:24.632256+0000","last_fullsized":"2024-02-14T17:46:24.632256+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:56.646656+0000","last_clean_scrub_stamp":"2024-02-14T17:44:56.646656+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:17:03.126359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001530008,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":80,"num_read_kb":75,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"68'12","reported_seq":47,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550956+0000","last_change":"2024-02-14T17:45:00.679791+0000","last_active":"2024-02-14T17:45:40.550956+0000","last_peered":"2024-02-14T17:45:40.550956+0000","last_clean":"2024-02-14T17:45:40.550956+0000","last_became_active":"2024-02-14T17:45:00.679446+0000","last_became_peered":"2024-02-14T17:45:00.679446+0000","last_unstale":"2024-02-14T17:45:40.550956+0000","last_undegraded":"2024-02-14T17:45:40.550956+0000","last_fullsized":"2024-02-14T17:45:40.550956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:54:32.231884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.333074+0000","last_change":"2024-02-14T17:45:02.703677+0000","last_active":"2024-02-14T17:45:40.333074+0000","last_peered":"2024-02-14T17:45:40.333074+0000","last_clean":"2024-02-14T17:45:40.333074+0000","last_became_active":"2024-02-14T17:45:02.703391+0000","last_became_peered":"2024-02-14T17:45:02.703391+0000","last_unstale":"2024-02-14T17:45:40.333074+0000","last_undegraded":"2024-02-14T17:45:40.333074+0000","last_fullsized":"2024-02-14T17:45:40.333074+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T22:15:56.079434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223384+0000","last_change":"2024-02-14T17:45:04.948584+0000","last_active":"2024-02-14T17:45:40.223384+0000","last_peered":"2024-02-14T17:45:40.223384+0000","last_clean":"2024-02-14T17:45:40.223384+0000","last_became_active":"2024-02-14T17:45:04.948214+0000","last_became_peered":"2024-02-14T17:45:04.948214+0000","last_unstale":"2024-02-14T17:45:40.223384+0000","last_undegraded":"2024-02-14T17:45:40.223384+0000","last_fullsized":"2024-02-14T17:45:40.223384+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:27:59.471019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224269+0000","last_change":"2024-02-14T17:44:58.667967+0000","last_active":"2024-02-14T17:45:40.224269+0000","last_peered":"2024-02-14T17:45:40.224269+0000","last_clean":"2024-02-14T17:45:40.224269+0000","last_became_active":"2024-02-14T17:44:58.667678+0000","last_became_peered":"2024-02-14T17:44:58.667678+0000","last_unstale":"2024-02-14T17:45:40.224269+0000","last_undegraded":"2024-02-14T17:45:40.224269+0000","last_fullsized":"2024-02-14T17:45:40.224269+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:39:33.937876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"68'19","reported_seq":65,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551823+0000","last_change":"2024-02-14T17:45:00.684543+0000","last_active":"2024-02-14T17:45:40.551823+0000","last_peered":"2024-02-14T17:45:40.551823+0000","last_clean":"2024-02-14T17:45:40.551823+0000","last_became_active":"2024-02-14T17:45:00.684197+0000","last_became_peered":"2024-02-14T17:45:00.684197+0000","last_unstale":"2024-02-14T17:45:40.551823+0000","last_undegraded":"2024-02-14T17:45:40.551823+0000","last_fullsized":"2024-02-14T17:45:40.551823+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:20:55.775840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550803+0000","last_change":"2024-02-14T17:45:02.696524+0000","last_active":"2024-02-14T17:45:40.550803+0000","last_peered":"2024-02-14T17:45:40.550803+0000","last_clean":"2024-02-14T17:45:40.550803+0000","last_became_active":"2024-02-14T17:45:02.696246+0000","last_became_peered":"2024-02-14T17:45:02.696246+0000","last_unstale":"2024-02-14T17:45:40.550803+0000","last_undegraded":"2024-02-14T17:45:40.550803+0000","last_fullsized":"2024-02-14T17:45:40.550803+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:31:38.961469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550383+0000","last_change":"2024-02-14T17:45:04.716852+0000","last_active":"2024-02-14T17:45:40.550383+0000","last_peered":"2024-02-14T17:45:40.550383+0000","last_clean":"2024-02-14T17:45:40.550383+0000","last_became_active":"2024-02-14T17:45:04.716543+0000","last_became_peered":"2024-02-14T17:45:04.716543+0000","last_unstale":"2024-02-14T17:45:40.550383+0000","last_undegraded":"2024-02-14T17:45:40.550383+0000","last_fullsized":"2024-02-14T17:45:40.550383+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:03:36.624467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332863+0000","last_change":"2024-02-14T17:44:58.663502+0000","last_active":"2024-02-14T17:45:40.332863+0000","last_peered":"2024-02-14T17:45:40.332863+0000","last_clean":"2024-02-14T17:45:40.332863+0000","last_became_active":"2024-02-14T17:44:58.663180+0000","last_became_peered":"2024-02-14T17:44:58.663180+0000","last_unstale":"2024-02-14T17:45:40.332863+0000","last_undegraded":"2024-02-14T17:45:40.332863+0000","last_fullsized":"2024-02-14T17:45:40.332863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:05:50.227869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223808+0000","last_change":"2024-02-14T17:45:00.681503+0000","last_active":"2024-02-14T17:45:40.223808+0000","last_peered":"2024-02-14T17:45:40.223808+0000","last_clean":"2024-02-14T17:45:40.223808+0000","last_became_active":"2024-02-14T17:45:00.681165+0000","last_became_peered":"2024-02-14T17:45:00.681165+0000","last_unstale":"2024-02-14T17:45:40.223808+0000","last_undegraded":"2024-02-14T17:45:40.223808+0000","last_fullsized":"2024-02-14T17:45:40.223808+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:38:36.501566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"68'8","reported_seq":51,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:18.732433+0000","last_change":"2024-02-14T17:45:02.698514+0000","last_active":"2024-02-14T17:46:18.732433+0000","last_peered":"2024-02-14T17:46:18.732433+0000","last_clean":"2024-02-14T17:46:18.732433+0000","last_became_active":"2024-02-14T17:45:02.698201+0000","last_became_peered":"2024-02-14T17:45:02.698201+0000","last_unstale":"2024-02-14T17:46:18.732433+0000","last_undegraded":"2024-02-14T17:46:18.732433+0000","last_fullsized":"2024-02-14T17:46:18.732433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:00:43.539317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.228614+0000","last_change":"2024-02-14T17:45:04.949846+0000","last_active":"2024-02-14T17:45:40.228614+0000","last_peered":"2024-02-14T17:45:40.228614+0000","last_clean":"2024-02-14T17:45:40.228614+0000","last_became_active":"2024-02-14T17:45:04.949579+0000","last_became_peered":"2024-02-14T17:45:04.949579+0000","last_unstale":"2024-02-14T17:45:40.228614+0000","last_undegraded":"2024-02-14T17:45:40.228614+0000","last_fullsized":"2024-02-14T17:45:40.228614+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T18:32:10.549937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"61'1","reported_seq":34,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551086+0000","last_change":"2024-02-14T17:44:58.673750+0000","last_active":"2024-02-14T17:45:40.551086+0000","last_peered":"2024-02-14T17:45:40.551086+0000","last_clean":"2024-02-14T17:45:40.551086+0000","last_became_active":"2024-02-14T17:44:58.673443+0000","last_became_peered":"2024-02-14T17:44:58.673443+0000","last_unstale":"2024-02-14T17:45:40.551086+0000","last_undegraded":"2024-02-14T17:45:40.551086+0000","last_fullsized":"2024-02-14T17:45:40.551086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:39:36.402781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"68'14","reported_seq":50,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222575+0000","last_change":"2024-02-14T17:45:00.677521+0000","last_active":"2024-02-14T17:45:40.222575+0000","last_peered":"2024-02-14T17:45:40.222575+0000","last_clean":"2024-02-14T17:45:40.222575+0000","last_became_active":"2024-02-14T17:45:00.677104+0000","last_became_peered":"2024-02-14T17:45:00.677104+0000","last_unstale":"2024-02-14T17:45:40.222575+0000","last_undegraded":"2024-02-14T17:45:40.222575+0000","last_fullsized":"2024-02-14T17:45:40.222575+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:46:56.073493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"68'8","reported_seq":51,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:18.732419+0000","last_change":"2024-02-14T17:45:02.702004+0000","last_active":"2024-02-14T17:46:18.732419+0000","last_peered":"2024-02-14T17:46:18.732419+0000","last_clean":"2024-02-14T17:46:18.732419+0000","last_became_active":"2024-02-14T17:45:02.701645+0000","last_became_peered":"2024-02-14T17:45:02.701645+0000","last_unstale":"2024-02-14T17:46:18.732419+0000","last_undegraded":"2024-02-14T17:46:18.732419+0000","last_fullsized":"2024-02-14T17:46:18.732419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:18:11.348488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222660+0000","last_change":"2024-02-14T17:45:04.713151+0000","last_active":"2024-02-14T17:45:40.222660+0000","last_peered":"2024-02-14T17:45:40.222660+0000","last_clean":"2024-02-14T17:45:40.222660+0000","last_became_active":"2024-02-14T17:45:04.712794+0000","last_became_peered":"2024-02-14T17:45:04.712794+0000","last_unstale":"2024-02-14T17:45:40.222660+0000","last_undegraded":"2024-02-14T17:45:40.222660+0000","last_fullsized":"2024-02-14T17:45:40.222660+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T02:57:17.701918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552635+0000","last_change":"2024-02-14T17:44:58.673011+0000","last_active":"2024-02-14T17:45:40.552635+0000","last_peered":"2024-02-14T17:45:40.552635+0000","last_clean":"2024-02-14T17:45:40.552635+0000","last_became_active":"2024-02-14T17:44:58.672321+0000","last_became_peered":"2024-02-14T17:44:58.672321+0000","last_unstale":"2024-02-14T17:45:40.552635+0000","last_undegraded":"2024-02-14T17:45:40.552635+0000","last_fullsized":"2024-02-14T17:45:40.552635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:02:40.531394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"68'18","reported_seq":61,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551270+0000","last_change":"2024-02-14T17:45:00.686285+0000","last_active":"2024-02-14T17:45:40.551270+0000","last_peered":"2024-02-14T17:45:40.551270+0000","last_clean":"2024-02-14T17:45:40.551270+0000","last_became_active":"2024-02-14T17:45:00.685970+0000","last_became_peered":"2024-02-14T17:45:00.685970+0000","last_unstale":"2024-02-14T17:45:40.551270+0000","last_undegraded":"2024-02-14T17:45:40.551270+0000","last_fullsized":"2024-02-14T17:45:40.551270+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:24:55.876345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222329+0000","last_change":"2024-02-14T17:45:02.700781+0000","last_active":"2024-02-14T17:45:40.222329+0000","last_peered":"2024-02-14T17:45:40.222329+0000","last_clean":"2024-02-14T17:45:40.222329+0000","last_became_active":"2024-02-14T17:45:02.700443+0000","last_became_peered":"2024-02-14T17:45:02.700443+0000","last_unstale":"2024-02-14T17:45:40.222329+0000","last_undegraded":"2024-02-14T17:45:40.222329+0000","last_fullsized":"2024-02-14T17:45:40.222329+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:59:23.639688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.332665+0000","last_change":"2024-02-14T17:45:04.714459+0000","last_active":"2024-02-14T17:45:40.332665+0000","last_peered":"2024-02-14T17:45:40.332665+0000","last_clean":"2024-02-14T17:45:40.332665+0000","last_became_active":"2024-02-14T17:45:04.714147+0000","last_became_peered":"2024-02-14T17:45:04.714147+0000","last_unstale":"2024-02-14T17:45:40.332665+0000","last_undegraded":"2024-02-14T17:45:40.332665+0000","last_fullsized":"2024-02-14T17:45:40.332665+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:00:54.615073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"68'8","reported_seq":51,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:18.732552+0000","last_change":"2024-02-14T17:45:02.697200+0000","last_active":"2024-02-14T17:46:18.732552+0000","last_peered":"2024-02-14T17:46:18.732552+0000","last_clean":"2024-02-14T17:46:18.732552+0000","last_became_active":"2024-02-14T17:45:02.696906+0000","last_became_peered":"2024-02-14T17:45:02.696906+0000","last_unstale":"2024-02-14T17:46:18.732552+0000","last_undegraded":"2024-02-14T17:46:18.732552+0000","last_fullsized":"2024-02-14T17:46:18.732552+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:10:19.873730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551176+0000","last_change":"2024-02-14T17:44:58.668323+0000","last_active":"2024-02-14T17:45:40.551176+0000","last_peered":"2024-02-14T17:45:40.551176+0000","last_clean":"2024-02-14T17:45:40.551176+0000","last_became_active":"2024-02-14T17:44:58.667925+0000","last_became_peered":"2024-02-14T17:44:58.667925+0000","last_unstale":"2024-02-14T17:45:40.551176+0000","last_undegraded":"2024-02-14T17:45:40.551176+0000","last_fullsized":"2024-02-14T17:45:40.551176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:25:31.365397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"68'15","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.224858+0000","last_change":"2024-02-14T17:45:00.690181+0000","last_active":"2024-02-14T17:45:40.224858+0000","last_peered":"2024-02-14T17:45:40.224858+0000","last_clean":"2024-02-14T17:45:40.224858+0000","last_became_active":"2024-02-14T17:45:00.689861+0000","last_became_peered":"2024-02-14T17:45:00.689861+0000","last_unstale":"2024-02-14T17:45:40.224858+0000","last_undegraded":"2024-02-14T17:45:40.224858+0000","last_fullsized":"2024-02-14T17:45:40.224858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:37:55.271333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.552136+0000","last_change":"2024-02-14T17:45:04.950845+0000","last_active":"2024-02-14T17:45:40.552136+0000","last_peered":"2024-02-14T17:45:40.552136+0000","last_clean":"2024-02-14T17:45:40.552136+0000","last_became_active":"2024-02-14T17:45:04.949811+0000","last_became_peered":"2024-02-14T17:45:04.949811+0000","last_unstale":"2024-02-14T17:45:40.552136+0000","last_undegraded":"2024-02-14T17:45:40.552136+0000","last_fullsized":"2024-02-14T17:45:40.552136+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:17:54.284167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.333070+0000","last_change":"2024-02-14T17:45:02.699552+0000","last_active":"2024-02-14T17:45:40.333070+0000","last_peered":"2024-02-14T17:45:40.333070+0000","last_clean":"2024-02-14T17:45:40.333070+0000","last_became_active":"2024-02-14T17:45:02.699282+0000","last_became_peered":"2024-02-14T17:45:02.699282+0000","last_unstale":"2024-02-14T17:45:40.333070+0000","last_undegraded":"2024-02-14T17:45:40.333070+0000","last_fullsized":"2024-02-14T17:45:40.333070+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T03:33:29.626427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222365+0000","last_change":"2024-02-14T17:44:58.668707+0000","last_active":"2024-02-14T17:45:40.222365+0000","last_peered":"2024-02-14T17:45:40.222365+0000","last_clean":"2024-02-14T17:45:40.222365+0000","last_became_active":"2024-02-14T17:44:58.668248+0000","last_became_peered":"2024-02-14T17:44:58.668248+0000","last_unstale":"2024-02-14T17:45:40.222365+0000","last_undegraded":"2024-02-14T17:45:40.222365+0000","last_fullsized":"2024-02-14T17:45:40.222365+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T05:24:28.786641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":48,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223299+0000","last_change":"2024-02-14T17:45:00.677662+0000","last_active":"2024-02-14T17:45:40.223299+0000","last_peered":"2024-02-14T17:45:40.223299+0000","last_clean":"2024-02-14T17:45:40.223299+0000","last_became_active":"2024-02-14T17:45:00.677303+0000","last_became_peered":"2024-02-14T17:45:00.677303+0000","last_unstale":"2024-02-14T17:45:40.223299+0000","last_undegraded":"2024-02-14T17:45:40.223299+0000","last_fullsized":"2024-02-14T17:45:40.223299+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T19:17:30.633703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223015+0000","last_change":"2024-02-14T17:45:04.714817+0000","last_active":"2024-02-14T17:45:40.223015+0000","last_peered":"2024-02-14T17:45:40.223015+0000","last_clean":"2024-02-14T17:45:40.223015+0000","last_became_active":"2024-02-14T17:45:04.714435+0000","last_became_peered":"2024-02-14T17:45:04.714435+0000","last_unstale":"2024-02-14T17:45:40.223015+0000","last_undegraded":"2024-02-14T17:45:40.223015+0000","last_fullsized":"2024-02-14T17:45:40.223015+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:09:21.320734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.225107+0000","last_change":"2024-02-14T17:45:02.702700+0000","last_active":"2024-02-14T17:45:40.225107+0000","last_peered":"2024-02-14T17:45:40.225107+0000","last_clean":"2024-02-14T17:45:40.225107+0000","last_became_active":"2024-02-14T17:45:02.702319+0000","last_became_peered":"2024-02-14T17:45:02.702319+0000","last_unstale":"2024-02-14T17:45:40.225107+0000","last_undegraded":"2024-02-14T17:45:40.225107+0000","last_fullsized":"2024-02-14T17:45:40.225107+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T23:48:02.062438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.550458+0000","last_change":"2024-02-14T17:44:58.672626+0000","last_active":"2024-02-14T17:45:40.550458+0000","last_peered":"2024-02-14T17:45:40.550458+0000","last_clean":"2024-02-14T17:45:40.550458+0000","last_became_active":"2024-02-14T17:44:58.672155+0000","last_became_peered":"2024-02-14T17:44:58.672155+0000","last_unstale":"2024-02-14T17:45:40.550458+0000","last_undegraded":"2024-02-14T17:45:40.550458+0000","last_fullsized":"2024-02-14T17:45:40.550458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:29:55.197436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"68'17","reported_seq":57,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223716+0000","last_change":"2024-02-14T17:45:00.679799+0000","last_active":"2024-02-14T17:45:40.223716+0000","last_peered":"2024-02-14T17:45:40.223716+0000","last_clean":"2024-02-14T17:45:40.223716+0000","last_became_active":"2024-02-14T17:45:00.679482+0000","last_became_peered":"2024-02-14T17:45:00.679482+0000","last_unstale":"2024-02-14T17:45:40.223716+0000","last_undegraded":"2024-02-14T17:45:40.223716+0000","last_fullsized":"2024-02-14T17:45:40.223716+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T00:01:38.324547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.223638+0000","last_change":"2024-02-14T17:45:04.713023+0000","last_active":"2024-02-14T17:45:40.223638+0000","last_peered":"2024-02-14T17:45:40.223638+0000","last_clean":"2024-02-14T17:45:40.223638+0000","last_became_active":"2024-02-14T17:45:04.712667+0000","last_became_peered":"2024-02-14T17:45:04.712667+0000","last_unstale":"2024-02-14T17:45:40.223638+0000","last_undegraded":"2024-02-14T17:45:40.223638+0000","last_fullsized":"2024-02-14T17:45:40.223638+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:03.684983+0000","last_clean_scrub_stamp":"2024-02-14T17:45:03.684983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T04:37:25.104842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"68'8","reported_seq":54,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:46:18.732408+0000","last_change":"2024-02-14T17:45:02.700504+0000","last_active":"2024-02-14T17:46:18.732408+0000","last_peered":"2024-02-14T17:46:18.732408+0000","last_clean":"2024-02-14T17:46:18.732408+0000","last_became_active":"2024-02-14T17:45:02.700200+0000","last_became_peered":"2024-02-14T17:45:02.700200+0000","last_unstale":"2024-02-14T17:46:18.732408+0000","last_undegraded":"2024-02-14T17:46:18.732408+0000","last_fullsized":"2024-02-14T17:46:18.732408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:45:01.677911+0000","last_clean_scrub_stamp":"2024-02-14T17:45:01.677911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T17:46:16.895507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.551966+0000","last_change":"2024-02-14T17:44:58.676721+0000","last_active":"2024-02-14T17:45:40.551966+0000","last_peered":"2024-02-14T17:45:40.551966+0000","last_clean":"2024-02-14T17:45:40.551966+0000","last_became_active":"2024-02-14T17:44:58.676400+0000","last_became_peered":"2024-02-14T17:44:58.676400+0000","last_unstale":"2024-02-14T17:45:40.551966+0000","last_undegraded":"2024-02-14T17:45:40.551966+0000","last_fullsized":"2024-02-14T17:45:40.551966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:57.653201+0000","last_clean_scrub_stamp":"2024-02-14T17:44:57.653201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-16T01:19:12.259015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"68'10","reported_seq":44,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-02-14T17:45:40.222936+0000","last_change":"2024-02-14T17:45:00.688284+0000","last_active":"2024-02-14T17:45:40.222936+0000","last_peered":"2024-02-14T17:45:40.222936+0000","last_clean":"2024-02-14T17:45:40.222936+0000","last_became_active":"2024-02-14T17:45:00.687843+0000","last_became_peered":"2024-02-14T17:45:00.687843+0000","last_unstale":"2024-02-14T17:45:40.222936+0000","last_undegraded":"2024-02-14T17:45:40.222936+0000","last_fullsized":"2024-02-14T17:45:40.222936+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-14T17:44:59.664031+0000","last_clean_scrub_stamp":"2024-02-14T17:44:59.664031+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T20:35:20.938287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":75,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234006,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32548,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712604,"statfs":{"total":95995035648,"available":95961706496,"internally_reserved":0,"allocated":1404928,"data_stored":929851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.57699999999999996}]},{"osd":2,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.52000000000000002}]},{"osd":3,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.71399999999999997}]},{"osd":4,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.80800000000000005}]},{"osd":5,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.75600000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495644,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712620,"statfs":{"total":95995035648,"available":95961722880,"internally_reserved":0,"allocated":1384448,"data_stored":928324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.66000000000000003}]},{"osd":1,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57699999999999996}]},{"osd":2,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.60699999999999998}]},{"osd":3,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.625},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.69499999999999995}]},{"osd":4,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.39400000000000002}]},{"osd":5,"last update":"Wed Feb 14 17:45:24 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed Feb 14 17:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":835584,"data_stored":338806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:46:19 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.35999999999999999,"15min":0.318},"min":{"1min":0.30299999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.373,"5min":0.36099999999999999,"15min":0.32900000000000001},"min":{"1min":0.25,"5min":0.191,"15min":0.191},"max":{"1min":0.51000000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.33200000000000002}]},{"osd":2,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36599999999999999,"15min":0.35299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.40799999999999997,"15min":0.39300000000000002},"min":{"1min":0.27900000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Feb 14 17:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47099999999999997,"15min":0.46200000000000002},"min":{"1min":0.29699999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.47299999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.36399999999999999}]},{"osd":4,"last update":"Wed Feb 14 17:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.40300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Wed Feb 14 17:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58999999999999997,"15min":0.59499999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.45500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63400000000000001}]},{"osd":6,"last update":"Wed Feb 14 17:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66000000000000003,"15min":0.65800000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.621},"min":{"1min":0.46100000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55700000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28396,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716756,"statfs":{"total":95995035648,"available":95965958144,"internally_reserved":0,"allocated":1413120,"data_stored":929607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 14 17:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.38200000000000001,"15min":0.36099999999999999},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.51800000000000002,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.505},{"interface":"front","average":{"1min":0.441,"5min":0.46100000000000002,"15min":0.42299999999999999},"min":{"1min":0.28000000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.57499999999999996,"5min":5.1980000000000004,"15min":5.1980000000000004},"last":0.56999999999999995}]},{"osd":2,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40500000000000003},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.435,"15min":0.42299999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":3.8580000000000001,"5min":3.8580000000000001,"15min":3.8580000000000001},"last":0.55200000000000005}]},{"osd":3,"last update":"Wed Feb 14 17:46:08 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.51600000000000001,"15min":0.48499999999999999},"min":{"1min":0.26400000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":9.1219999999999999,"15min":9.1219999999999999},"last":0.442},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.44700000000000001,"15min":0.46300000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.59599999999999997,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.53700000000000003}]},{"osd":4,"last update":"Wed Feb 14 17:45:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.6459999999999999,"5min":3.6459999999999999,"15min":3.6459999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Feb 14 17:45:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.58799999999999997,"15min":0.57199999999999995},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.5110000000000001,"5min":3.5110000000000001,"15min":3.5110000000000001},"last":0.755},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70399999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed Feb 14 17:46:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80500000000000005,"5min":3.6030000000000002,"15min":3.6030000000000002},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.94699999999999995,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.66400000000000003}]},{"osd":7,"last update":"Wed Feb 14 17:45:44 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":4.4130000000000003,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.8959999999999999,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":0.68500000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":806912,"data_stored":338620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.38500000000000001,"15min":0.38400000000000001},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.52000000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.51600000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.38}]},{"osd":1,"last update":"Wed Feb 14 17:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.24099999999999999,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.49099999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42499999999999999,"15min":0.41899999999999998},"min":{"1min":0.26500000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.54000000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.36599999999999999}]},{"osd":3,"last update":"Wed Feb 14 17:46:07 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45300000000000001,"15min":0.46200000000000002},"min":{"1min":0.32300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.32100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.56399999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.26300000000000001}]},{"osd":4,"last update":"Wed Feb 14 17:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.57399999999999995}]},{"osd":5,"last update":"Wed Feb 14 17:46:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.59599999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.56499999999999995}]},{"osd":6,"last update":"Wed Feb 14 17:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Feb 14 17:45:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.63700000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":827392,"data_stored":338847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.38700000000000001,"15min":0.38300000000000001},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41899999999999998,"15min":0.41599999999999998},"min":{"1min":0.24399999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.40300000000000002}]},{"osd":1,"last update":"Wed Feb 14 17:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.434,"15min":0.442},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.42899999999999999,"15min":0.433},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.44}]},{"osd":2,"last update":"Wed Feb 14 17:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46200000000000002,"15min":0.46000000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.439,"15min":0.439},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Feb 14 17:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55600000000000005,"15min":0.55700000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76700000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.84199999999999997}]},{"osd":5,"last update":"Wed Feb 14 17:46:01 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59199999999999997,"15min":0.58599999999999997},"min":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.874},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed Feb 14 17:46:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.625,"15min":0.624},"min":{"1min":0.45600000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Wed Feb 14 17:45:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.83099999999999996}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":819200,"data_stored":339037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":0.59699999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75800000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.65900000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58699999999999997}]},{"osd":1,"last update":"Wed Feb 14 17:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.64800000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55500000000000005,"15min":0.56000000000000005},"min":{"1min":0.35799999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70599999999999996,"5min":0.87,"15min":0.87},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Feb 14 17:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.68000000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54800000000000004,"15min":0.55300000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66200000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.52400000000000002}]},{"osd":3,"last update":"Wed Feb 14 17:45:30 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56200000000000006,"15min":0.56899999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66800000000000004,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.61499999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48099999999999998}]},{"osd":5,"last update":"Wed Feb 14 17:45:54 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.435,"15min":0.435},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.60799999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.46500000000000002,"15min":0.45700000000000002},"min":{"1min":0.38800000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.44400000000000001}]},{"osd":6,"last update":"Wed Feb 14 17:46:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.46600000000000003,"15min":0.45400000000000001},"min":{"1min":0.38700000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.44800000000000001,"15min":0.434},"min":{"1min":0.34599999999999997,"5min":0.311,"15min":0.311},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.36599999999999999}]},{"osd":7,"last update":"Wed Feb 14 17:45:49 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.34399999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757280,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":794624,"data_stored":337896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 14 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.69099999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.65100000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70199999999999996}]},{"osd":1,"last update":"Wed Feb 14 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.68600000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Wed Feb 14 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68700000000000006,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58499999999999996}]},{"osd":3,"last update":"Wed Feb 14 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.52200000000000002,"15min":0.52600000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71199999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.55600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.751,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Feb 14 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.36799999999999999,"15min":0.35999999999999999},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.40400000000000003}]},{"osd":6,"last update":"Wed Feb 14 17:46:23 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.65700000000000003,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.41599999999999998,"15min":0.40799999999999997},"min":{"1min":0.311,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.60199999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45400000000000001}]},{"osd":7,"last update":"Wed Feb 14 17:45:47 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.63100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-14T17:46:27.271 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-14T17:46:27.271 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-14T17:46:27.271 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-14T17:46:27.271 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1 shell --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 -- ceph health --format=json 2024-02-14T17:46:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:26 smithi179 ceph-mon[33633]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-14T17:46:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:26 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-14T17:46:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:26 smithi179 ceph-mon[33633]: Deploying daemon grafana.a on smithi179 2024-02-14T17:46:27.520 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/mon.a/config 2024-02-14T17:46:27.900 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:27 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:27.557Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000158204s 2024-02-14T17:46:28.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:27 smithi179 ceph-mon[33633]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:28.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:27 smithi179 ceph-mon[33633]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:46:28.388 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:27 smithi100 ceph-mon[39557]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:28.389 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:27 smithi100 ceph-mon[39557]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:46:28.389 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:27 smithi100 ceph-mon[30138]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:28.389 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:27 smithi100 ceph-mon[30138]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-14T17:46:29.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 -- 172.21.15.100:0/1569159414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8101fe0 msgr2=0x7f33a810e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:29.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 --2- 172.21.15.100:0/1569159414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8101fe0 0x7f33a810e1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f339c009990 tx=0x7f339c009ca0 comp rx=0 tx=0).stop 2024-02-14T17:46:29.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 -- 172.21.15.100:0/1569159414 shutdown_connections 2024-02-14T17:46:29.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 --2- 172.21.15.100:0/1569159414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a81016c0 0x7f33a8101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:29.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 --2- 172.21.15.100:0/1569159414 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33a810e700 0x7f33a8110af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:29.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 --2- 172.21.15.100:0/1569159414 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8101fe0 0x7f33a810e1c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:29.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 -- 172.21.15.100:0/1569159414 >> 172.21.15.100:0/1569159414 conn(0x7f33a8077bd0 msgr2=0x7f33a8077fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:29.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 -- 172.21.15.100:0/1569159414 shutdown_connections 2024-02-14T17:46:29.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 -- 172.21.15.100:0/1569159414 wait complete. 2024-02-14T17:46:29.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 Processor -- start 2024-02-14T17:46:29.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.294+0000 7f33ae9a2700 1 -- start start 2024-02-14T17:46:29.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33ae9a2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a81016c0 0x7f33a819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33ae9a2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8101fe0 0x7f33a819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33ae9a2700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33a810e700 0x7f33a8199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33ae9a2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8199ec0 con 0x7f33a8101fe0 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33ae9a2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a819a030 con 0x7f33a81016c0 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33ae9a2700 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a819a1f0 con 0x7f33a810e700 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33a77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8101fe0 0x7f33a819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33a7fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a81016c0 0x7f33a819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33acf3f700 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33a810e700 0x7f33a8199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33a77fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8101fe0 0x7f33a819fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36218/0 (socket says 172.21.15.100:36218) 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33a77fe700 1 -- 172.21.15.100:0/2795169609 learned_addr learned my addr 172.21.15.100:0/2795169609 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33a77fe700 1 -- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33a810e700 msgr2=0x7f33a8199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33a77fe700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33a810e700 0x7f33a8199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:29.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33a77fe700 1 -- 172.21.15.100:0/2795169609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a81016c0 msgr2=0x7f33a819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:29.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33a77fe700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a81016c0 0x7f33a819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:29.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.295+0000 7f33a77fe700 1 -- 172.21.15.100:0/2795169609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a0009c40 con 0x7f33a8101fe0 2024-02-14T17:46:29.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.296+0000 7f33a77fe700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8101fe0 0x7f33a819fbd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f339c004760 tx=0x7f339c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:29.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.296+0000 7f33a57fa700 1 -- 172.21.15.100:0/2795169609 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c01d070 con 0x7f33a8101fe0 2024-02-14T17:46:29.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.296+0000 7f33ae9a2700 1 -- 172.21.15.100:0/2795169609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339c0095f0 con 0x7f33a8101fe0 2024-02-14T17:46:29.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.296+0000 7f33a57fa700 1 -- 172.21.15.100:0/2795169609 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f339c005a00 con 0x7f33a8101fe0 2024-02-14T17:46:29.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.296+0000 7f33a57fa700 1 -- 172.21.15.100:0/2795169609 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c005020 con 0x7f33a8101fe0 2024-02-14T17:46:29.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.296+0000 7f33ae9a2700 1 -- 172.21.15.100:0/2795169609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a81a64f0 con 0x7f33a8101fe0 2024-02-14T17:46:29.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.297+0000 7f33ae9a2700 1 -- 172.21.15.100:0/2795169609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a8109920 con 0x7f33a8101fe0 2024-02-14T17:46:29.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.297+0000 7f33a57fa700 1 -- 172.21.15.100:0/2795169609 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f339c004830 con 0x7f33a8101fe0 2024-02-14T17:46:29.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.298+0000 7f33a57fa700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f339006dac0 0x7f339006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T17:46:29.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.298+0000 7f33a7fff700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f339006dac0 0x7f339006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T17:46:29.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.298+0000 7f33a57fa700 1 -- 172.21.15.100:0/2795169609 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f339c08e7c0 con 0x7f33a8101fe0 2024-02-14T17:46:29.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.298+0000 7f33a7fff700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f339006dac0 0x7f339006ff80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3398009c60 tx=0x7f3398009260 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T17:46:29.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.302+0000 7f33a57fa700 1 -- 172.21.15.100:0/2795169609 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f339c057bd0 con 0x7f33a8101fe0 2024-02-14T17:46:29.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.544+0000 7f33ae9a2700 1 -- 172.21.15.100:0/2795169609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f33a8068930 con 0x7f33a8101fe0 2024-02-14T17:46:29.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.544+0000 7f33a57fa700 1 -- 172.21.15.100:0/2795169609 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f339c05b290 con 0x7f33a8101fe0 2024-02-14T17:46:29.546 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-02-14T17:46:29.546 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-14T17:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 -- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f339006dac0 msgr2=0x7f339006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f339006dac0 0x7f339006ff80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3398009c60 tx=0x7f3398009260 comp rx=0 tx=0).stop 2024-02-14T17:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 -- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8101fe0 msgr2=0x7f33a819fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T17:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8101fe0 0x7f33a819fbd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f339c004760 tx=0x7f339c004790 comp rx=0 tx=0).stop 2024-02-14T17:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 -- 172.21.15.100:0/2795169609 shutdown_connections 2024-02-14T17:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33a81016c0 0x7f33a819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f339006dac0 0x7f339006ff80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f33a810e700 0x7f33a8199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 --2- 172.21.15.100:0/2795169609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33a8101fe0 0x7f33a819fbd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T17:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 -- 172.21.15.100:0/2795169609 >> 172.21.15.100:0/2795169609 conn(0x7f33a8077bd0 msgr2=0x7f33a810fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T17:46:29.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 -- 172.21.15.100:0/2795169609 shutdown_connections 2024-02-14T17:46:29.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-02-14T17:46:29.546+0000 7f33ae9a2700 1 -- 172.21.15.100:0/2795169609 wait complete. 2024-02-14T17:46:30.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:29 smithi100 ceph-mon[39557]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:30.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2795169609' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T17:46:30.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:29 smithi100 ceph-mon[30138]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:30.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2795169609' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T17:46:30.198 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-14T17:46:30.198 INFO:tasks.cephadm:Setup complete, yielding 2024-02-14T17:46:30.198 INFO:teuthology.run_tasks:Running task workunit... 2024-02-14T17:46:30.211 INFO:tasks.workunit:Pulling workunits from ref 8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:46:30.211 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-14T17:46:30.211 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-14T17:46:30.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T17:46:30.227 INFO:teuthology.orchestra.run.smithi100.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-14T17:46:30.228 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-14T17:46:30.282 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-14T17:46:30.282 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-14T17:46:30.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:29 smithi179 ceph-mon[33633]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:30.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2795169609' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-14T17:46:30.337 INFO:tasks.workunit:timeout=3h 2024-02-14T17:46:30.337 INFO:tasks.workunit:cleanup=True 2024-02-14T17:46:30.338 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8727ce939fc044a98f215e29696bb1433f5758d1 2024-02-14T17:46:30.392 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-14T17:46:30.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:46:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:31 smithi179 ceph-mon[33633]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:31 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:31.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:31 smithi100 ceph-mon[30138]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:31.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:31 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:31.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:31 smithi100 ceph-mon[39557]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:31.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:31 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:32.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:46:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:46:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:32 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:32.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:32 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:32.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:32 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:33 smithi100 ceph-mon[30138]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:33.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:33 smithi100 ceph-mon[39557]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:33.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:33 smithi179 ceph-mon[33633]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:35 smithi100 ceph-mon[39557]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:35.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:35 smithi100 ceph-mon[30138]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:35.651 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:35 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:35.560Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002729588s 2024-02-14T17:46:35.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:35 smithi179 ceph-mon[33633]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:37 smithi179 ceph-mon[33633]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:37.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:37 smithi100 ceph-mon[39557]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:37 smithi100 ceph-mon[30138]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:39.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:39 smithi100 ceph-mon[30138]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:39.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:39 smithi100 ceph-mon[39557]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:39.756 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:39 smithi179 ceph-mon[33633]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:39.756 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:39 smithi179 systemd[1]: Starting Ceph grafana.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:46:40.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:39 smithi179 podman[66319]: 2024-02-14 17:46:39.657228298 +0000 UTC m=+0.017138042 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-02-14T17:46:40.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:39 smithi179 podman[66319]: 2024-02-14 17:46:39.780859591 +0000 UTC m=+0.140769328 container create a4c562e237e55df4a9ec4d5087751c691061da7f0901fb723c6883b5c2d29639 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8, description=Ceph Grafana Container, io.buildah.version=1.32.0, architecture=x86_64, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, maintainer=Paul Cuzner ) 2024-02-14T17:46:40.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:39 smithi179 podman[66319]: 2024-02-14 17:46:39.992828426 +0000 UTC m=+0.352738168 container init a4c562e237e55df4a9ec4d5087751c691061da7f0901fb723c6883b5c2d29639 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, version=8.5, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, name=ubi8, io.openshift.expose-services=, io.openshift.tags=base rhel8, io.buildah.version=1.32.0, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-02-14T17:46:40.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:39 smithi179 podman[66319]: 2024-02-14 17:46:39.998467607 +0000 UTC m=+0.358377348 container start a4c562e237e55df4a9ec4d5087751c691061da7f0901fb723c6883b5c2d29639 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a, io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, distribution-scope=public, description=Ceph Grafana Container, vendor=Red Hat, Inc., io.buildah.version=1.32.0, com.redhat.component=ubi8-container, release=239.1651231664, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8, vcs-type=git, build-date=2022-04-29T11:48:11.055897) 2024-02-14T17:46:40.345 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 bash[66319]: a4c562e237e55df4a9ec4d5087751c691061da7f0901fb723c6883b5c2d29639 2024-02-14T17:46:40.345 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 systemd[1]: Started Ceph grafana.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:46:40.345 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.150905738Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-14T17:46:40.345 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151100865Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-14T17:46:40.345 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151124712Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-14T17:46:40.345 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151132337Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-14T17:46:40.345 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151138605Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-14T17:46:40.345 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151144494Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151150456Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151156216Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151162272Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151168033Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151173639Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151179036Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151184699Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=settings t=2024-02-14T17:46:40.151190225Z level=info msg="App mode production" 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=sqlstore t=2024-02-14T17:46:40.151234469Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=sqlstore t=2024-02-14T17:46:40.151250367Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.15206009Z level=info msg="Starting DB migrations" 2024-02-14T17:46:40.346 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.152142692Z level=info msg="Executing migration" id="create migration_log table" 2024-02-14T17:46:40.347 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.152645179Z level=info msg="Executing migration" id="create user table" 2024-02-14T17:46:40.347 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.153076914Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-14T17:46:40.347 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.153544683Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-14T17:46:40.347 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.153983592Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-14T17:46:40.347 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.154556728Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-14T17:46:40.347 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.154938076Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-14T17:46:40.347 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.155640418Z level=info msg="Executing migration" id="create user table v2" 2024-02-14T17:46:40.347 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.156014021Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.156775377Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.157197808Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.15758717Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.157946362Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.158414719Z level=info msg="Executing migration" id="Update user table charset" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.158736765Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.159145037Z level=info msg="Executing migration" id="Add missing user data" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.159547214Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.159966029Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.16040447Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.1608458Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.161997295Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-14T17:46:40.348 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.162644707Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.163287304Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.163686609Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.164075367Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.164480732Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.164748174Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.165124665Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.165529462Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.165886022Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.166280275Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.166901788Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.167323148Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.167724513Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.168119677Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.168517817Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-14T17:46:40.349 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.168919651Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.169270271Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.169615002Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.169924228Z level=info msg="Executing migration" id="create star table" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.170302541Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.170708059Z level=info msg="Executing migration" id="create org table v1" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.171095086Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.171499681Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.171865588Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.172277237Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.17268598Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.17307054Z level=info msg="Executing migration" id="Update org table charset" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.173373134Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.173645331Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.173930964Z level=info msg="Executing migration" id="create dashboard table" 2024-02-14T17:46:40.350 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.174338537Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-14T17:46:40.351 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.174727794Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-14T17:46:40.351 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.175126179Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-14T17:46:40.351 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.175537836Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-14T17:46:40.351 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.176405123Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-14T17:46:40.351 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.177219226Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-14T17:46:40.351 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.178099054Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-14T17:46:40.351 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.178893588Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-14T17:46:40.352 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.17968731Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-14T17:46:40.352 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.180644734Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-14T17:46:40.352 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.181232741Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-14T17:46:40.352 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.181647599Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-14T17:46:40.352 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.181934652Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-14T17:46:40.352 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.182456622Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-14T17:46:40.352 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.183280809Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-14T17:46:40.352 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.184129799Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-14T17:46:40.352 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.185014686Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.185688168Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.186454261Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.186917966Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.187285352Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.187593607Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.188167092Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.188952556Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.189906689Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.190625918Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.19148789Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.192007771Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.192426599Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.193393102Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-14T17:46:40.353 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.194140088Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.195163368Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.196129177Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.197066365Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.19783793Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.198699081Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.19956223Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.202271265Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.204904452Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.207710062Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.210471673Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.213260564Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-14T17:46:40.354 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.216195064Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.218856915Z level=info msg="Executing migration" id="create data_source table" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.221353353Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.221790092Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.224032736Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.226630652Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.229453359Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.232376888Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.235104112Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.237621776Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.240465752Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.243144736Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-14T17:46:40.355 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.245859586Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.248589979Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.251286438Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.253810209Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.256582629Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.259134668Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.261679109Z level=info msg="Executing migration" id="Add uid column" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.264366638Z level=info msg="Executing migration" id="Update uid value" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.264683036Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.265452261Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.265880673Z level=info msg="Executing migration" id="create api_key table" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.266288394Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.267009061Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.267838482Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-14T17:46:40.356 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.268462256Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.26923797Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.269633407Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.270324312Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.271353079Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.271737116Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.272492307Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.272925572Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.273347899Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.274139316Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.274994701Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.275704314Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.276264646Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.277062507Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-14T17:46:40.357 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.277906303Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.278720865Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.27929238Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.280086366Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.280839911Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.281795674Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.282387743Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.283159249Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.28593668Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.288559618Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.291435026Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.294271194Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.296804132Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.299606054Z level=info msg="Executing migration" id="create quota table v1" 2024-02-14T17:46:40.358 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.30001046Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.302729973Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.3056716Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.308161583Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.308607512Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.311220667Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.311681115Z level=info msg="Executing migration" id="create session table" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.314061084Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.316619644Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.316957702Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.319782173Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.322423532Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.324894741Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.327545555Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.330056661Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.330405492Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-14T17:46:40.359 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.333160392Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-14T17:46:40.360 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.335535643Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-14T17:46:40.360 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.337929598Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-14T17:46:40.360 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.338288612Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-14T17:46:40.360 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.3409165Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-14T17:46:40.360 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.343644543Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-14T17:46:40.360 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.346059542Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-14T17:46:40.360 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.34651987Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-14T17:46:40.360 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.349215405Z level=info msg="Executing migration" id="create alert table v1" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.351568784Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.354173837Z level=info msg="Executing migration" id="add index alert state" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.354614501Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.357201997Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.358044921Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.358814013Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.359239833Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.360698933Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.361460065Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.362243613Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.362579546Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-14T17:46:40.596 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.363281945Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-14T17:46:40.597 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.363786426Z level=info msg="Executing migration" id="Add column is_default" 2024-02-14T17:46:40.597 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.364456505Z level=info msg="Executing migration" id="Add column frequency" 2024-02-14T17:46:40.597 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.365249135Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-14T17:46:40.597 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.36590153Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-14T17:46:40.597 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.366616159Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-14T17:46:40.597 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.36746695Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-14T17:46:40.597 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.368111178Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.368418727Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.369341418Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.369951416Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.370754065Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.371178483Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.37201841Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.372800047Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.373611872Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.373942661Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.374760424Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.375693484Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-14T17:46:40.598 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.376500107Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.376815801Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.377633532Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.378434828Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.379359306Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.37993372Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.380876854Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.381517744Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.382339586Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.382869316Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.385539289Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.388301798Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.390979618Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.391477883Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-14T17:46:40.599 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.39436459Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.397118126Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.399424996Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.399903467Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.402733173Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.405262018Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.407908746Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.408281086Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.41117817Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.414016554Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.416582186Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.417327626Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.420274646Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.422695449Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-14T17:46:40.600 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.425123982Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.425859586Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.428446256Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.431315052Z level=info msg="Executing migration" id="Move region to single row" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.433692973Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.43419547Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.436571882Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.439040633Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.44150415Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.441976378Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.444434809Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.446915259Z level=info msg="Executing migration" id="create test_data table" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.449492451Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.449945414Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-14T17:46:40.601 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.450723985Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-14T17:46:40.602 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.451497056Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-14T17:46:40.602 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.452390319Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-14T17:46:40.603 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.452957669Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-14T17:46:40.603 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.45376761Z level=info msg="Executing migration" id="create team table" 2024-02-14T17:46:40.603 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.454563882Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-14T17:46:40.603 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.455321249Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-14T17:46:40.603 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.455791109Z level=info msg="Executing migration" id="create team member table" 2024-02-14T17:46:40.603 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.456586483Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-14T17:46:40.603 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.457328844Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-14T17:46:40.603 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.458142234Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-14T17:46:40.603 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.458583541Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.459362928Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.460149577Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.46096598Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.461409029Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.462216371Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.462952947Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.463467392Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.463993186Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.464833486Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.465687846Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.466514347Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.467360075Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-14T17:46:40.604 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.468217891Z level=info msg="Executing migration" id="create tag table" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.469116776Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.469789754Z level=info msg="Executing migration" id="create login attempt table" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.470697566Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.471410575Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.472303138Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.474227099Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.475061389Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.476000184Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.476722142Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.47760722Z level=info msg="Executing migration" id="create user auth table" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.478238853Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.478703114Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.481498282Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.482509293Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.484250679Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-14T17:46:40.605 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.487173575Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.490029018Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.49261388Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.495022746Z level=info msg="Executing migration" id="create server_lock table" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.497627053Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.500310041Z level=info msg="Executing migration" id="create user auth token table" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.502938114Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.505769703Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.50626189Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.508980332Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.51172893Z level=info msg="Executing migration" id="create cache_data table" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.514300406Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.514793239Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.51775039Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.520215188Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-14T17:46:40.606 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.520672597Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.520939869Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.522739283Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.525277261Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.525700074Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.525954965Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.526354932Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.526752384Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.527139143Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.527549016Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.528315161Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.528730908Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.529185827Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.529585518Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.529971199Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-14T17:46:40.607 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.530390302Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.530674736Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.531069442Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.531484107Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.531858742Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.532285801Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.533062007Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.533466132Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.533871551Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.53725254Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.540634316Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.541055968Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.541471668Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-14T17:46:40.608 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.54229176Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.542691091Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.543116178Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.543530706Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.543930675Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.544225864Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.545013187Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.545833278Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.546653193Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.547055572Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.547483624Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.548275769Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.549055922Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.549494301Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-14T17:46:40.609 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.55030577Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.551116966Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.551423283Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.551848214Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.554306147Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.554778408Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.555033904Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.555881659Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.556712792Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.557796068Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.558632046Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.559476991Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.559771144Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.560150347Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.560955709Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-14T17:46:40.610 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.561256706Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.562151656Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.565002903Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.567771082Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.570353381Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.570913862Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.573728406Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.576265325Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.579112606Z level=info msg="Executing migration" id="create alert_image table" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.579586035Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.58238111Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.584940892Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.585378863Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-14T17:46:40.611 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.585804092Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.585877863Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.588494533Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.588873452Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.589308112Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.589424586Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.589530035Z level=info msg="alerts found to migrate" alerts=0 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.590143086Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.59060106Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.591365784Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.592274277Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.592937841Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.593707224Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.59447929Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-14T17:46:40.612 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.595243912Z level=info msg="Executing migration" id="create data_keys table" 2024-02-14T17:46:40.613 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.59603364Z level=info msg="Executing migration" id="create secrets table" 2024-02-14T17:46:40.613 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.596463572Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-14T17:46:40.613 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.600772281Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-14T17:46:40.820 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:46:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:41.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.603371466Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-14T17:46:41.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.60591391Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-14T17:46:41.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.610193832Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-14T17:46:41.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.614451979Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-14T17:46:41.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.615171133Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-14T17:46:41.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.615896566Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-14T17:46:41.072 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.616613022Z level=info msg="Executing migration" id="create permission table" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.617411928Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.618084984Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.618793219Z level=info msg="Executing migration" id="create role table" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.6192021Z level=info msg="Executing migration" id="add column display_name" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.620147352Z level=info msg="Executing migration" id="add column group_name" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.621041841Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.621858261Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.62247124Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.623203422Z level=info msg="Executing migration" id="create team role table" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.624017406Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.624629637Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-14T17:46:41.073 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.625092263Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-14T17:46:41.074 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.625813139Z level=info msg="Executing migration" id="create user role table" 2024-02-14T17:46:41.074 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.626599464Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-14T17:46:41.074 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.627397193Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-14T17:46:41.074 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.62784061Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.628643174Z level=info msg="Executing migration" id="create builtin role table" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.629545102Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.630192279Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.63063576Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.631661921Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.632427074Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.633279776Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.633792885Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.634633462Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.635238181Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.635657176Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.636619001Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.637005658Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.637900873Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-14T17:46:41.075 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.640312876Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.64293438Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.643394063Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.646170101Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.648876041Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.651415187Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.651866971Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.654724148Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.657163308Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.660011147Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.660478782Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-14T17:46:41.076 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.663225912Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.665689478Z level=info msg="Executing migration" id="add correlation config column" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.668478858Z level=info msg="Executing migration" id="create entity_events table" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.669065468Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.671817344Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.671914246Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.674613265Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.674716512Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.677127749Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.677589551Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.68022169Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.680834443Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.682578377Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-14T17:46:41.077 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.685045878Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.685626632Z level=info msg="Executing migration" id="Drop public config table" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.68802071Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.688467986Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.690509667Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.691013587Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.693252421Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.696407588Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.699068539Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.701608092Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.704015816Z level=info msg="Executing migration" id="add share column" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.705034527Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.707711367Z level=info msg="Executing migration" id="create file table" 2024-02-14T17:46:41.078 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.708378506Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.709154021Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.709645146Z level=info msg="Executing migration" id="create file_meta table" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.710314296Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.711096791Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.711850147Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.712359028Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.713134215Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.713919054Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.714970993Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.715312464Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.716192023Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.716777943Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.717519669Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.717988173Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.719089263Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-14T17:46:41.079 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.72012722Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.720882812Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.730780275Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.731289872Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.732139269Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.735656875Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.736431136Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.736817112Z level=info msg="Executing migration" id="create folder table" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.737647027Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.738463491Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.739290734Z level=info msg="Executing migration" id="Update folder title length" 2024-02-14T17:46:41.080 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.739753115Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-14T17:46:41.081 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=migrator t=2024-02-14T17:46:40.740603183Z level=info msg="migrations completed" performed=485 skipped=0 duration=588.471298ms 2024-02-14T17:46:41.081 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=sqlstore t=2024-02-14T17:46:40.741020113Z level=info msg="Created default organization" 2024-02-14T17:46:41.081 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=plugin.loader t=2024-02-14T17:46:40.795279451Z level=info msg="Plugin registered" pluginID=input 2024-02-14T17:46:41.081 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=plugin.loader t=2024-02-14T17:46:40.823753092Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-14T17:46:41.081 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=plugin.loader t=2024-02-14T17:46:40.823783712Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-14T17:46:41.081 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=secrets t=2024-02-14T17:46:40.823827576Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-14T17:46:41.081 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=query_data t=2024-02-14T17:46:40.825876928Z level=info msg="Query Service initialization" 2024-02-14T17:46:41.081 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=live.push_http t=2024-02-14T17:46:40.828334358Z level=info msg="Live Push Gateway initialization" 2024-02-14T17:46:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[30138]: Deploying daemon node-exporter.a on smithi100 2024-02-14T17:46:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[30138]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[39557]: Deploying daemon node-exporter.a on smithi100 2024-02-14T17:46:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[39557]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:41 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:41.401 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:41 smithi100 systemd[1]: Starting Ceph node-exporter.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:46:41.401 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:41 smithi100 bash[77809]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-14T17:46:41.432 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.432 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.432 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.432 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.432 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-mon[33633]: Deploying daemon node-exporter.a on smithi100 2024-02-14T17:46:41.432 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-mon[33633]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:41.433 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:41.433 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:41.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:46:41.821 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=infra.usagestats.collector t=2024-02-14T17:46:41.431742589Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-14T17:46:41.821 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=provisioning.datasources t=2024-02-14T17:46:41.432817382Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-14T17:46:41.822 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=provisioning.datasources t=2024-02-14T17:46:41.443938081Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-14T17:46:41.822 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=provisioning.alerting t=2024-02-14T17:46:41.454794984Z level=info msg="starting to provision alerting" 2024-02-14T17:46:41.822 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=provisioning.alerting t=2024-02-14T17:46:41.454813687Z level=info msg="finished to provision alerting" 2024-02-14T17:46:41.822 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=ngalert.state.manager t=2024-02-14T17:46:41.454884314Z level=info msg="Warming state cache for startup" 2024-02-14T17:46:41.822 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=grafanaStorageLogger t=2024-02-14T17:46:41.454918725Z level=info msg="storage starting" 2024-02-14T17:46:41.822 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=ngalert.state.manager t=2024-02-14T17:46:41.455056122Z level=info msg="State cache has been initialized" states=0 duration=171.33µs 2024-02-14T17:46:41.822 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=ngalert.multiorg.alertmanager t=2024-02-14T17:46:41.455142355Z level=info msg="starting MultiOrg Alertmanager" 2024-02-14T17:46:41.822 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=ticker t=2024-02-14T17:46:41.455180566Z level=info msg=starting first_tick=2024-02-14T17:46:50Z 2024-02-14T17:46:41.822 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=http.server t=2024-02-14T17:46:41.457019716Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-14T17:46:41.822 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:46:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=sqlstore.transactions t=2024-02-14T17:46:41.480727647Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-14T17:46:42.150 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:41 smithi100 bash[77809]: Getting image source signatures 2024-02-14T17:46:42.150 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:41 smithi100 bash[77809]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-14T17:46:42.150 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:41 smithi100 bash[77809]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-14T17:46:42.150 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:41 smithi100 bash[77809]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-14T17:46:42.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:42 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:42 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:42 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:43 smithi179 ceph-mon[33633]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:43 smithi100 ceph-mon[30138]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:43 smithi100 ceph-mon[39557]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:43.651 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:43 smithi100 bash[77809]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-14T17:46:43.651 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:43 smithi100 bash[77809]: Writing manifest to image destination 2024-02-14T17:46:44.034 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:43 smithi100 podman[77809]: 2024-02-14 17:46:43.682806779 +0000 UTC m=+2.380222623 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-14T17:46:44.034 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:43 smithi100 podman[77809]: 2024-02-14 17:46:43.78113362 +0000 UTC m=+2.478549458 container create 7eabaaf7305d5ac2036bd05d074e84b2944a040542f605a8d0873cfeb7dc073e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T17:46:44.034 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 podman[77809]: 2024-02-14 17:46:44.033438609 +0000 UTC m=+2.730854448 container init 7eabaaf7305d5ac2036bd05d074e84b2944a040542f605a8d0873cfeb7dc073e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T17:46:44.401 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 podman[77809]: 2024-02-14 17:46:44.039280085 +0000 UTC m=+2.736695925 container start 7eabaaf7305d5ac2036bd05d074e84b2944a040542f605a8d0873cfeb7dc073e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T17:46:44.401 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.043Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-14T17:46:44.401 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.043Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-14T17:46:44.401 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.043Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-14T17:46:44.401 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.043Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-14T17:46:44.401 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-14T17:46:44.401 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-14T17:46:44.401 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-14T17:46:44.402 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=arp 2024-02-14T17:46:44.402 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-14T17:46:44.402 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-14T17:46:44.402 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-14T17:46:44.402 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-14T17:46:44.402 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-14T17:46:44.402 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-14T17:46:44.402 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-14T17:46:44.402 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=edac 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-14T17:46:44.403 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=os 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-14T17:46:44.404 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.044Z caller=node_exporter.go:117 level=info collector=stat 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=node_exporter.go:117 level=info collector=time 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=node_exporter.go:117 level=info collector=uname 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a[77860]: ts=2024-02-14T17:46:44.045Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 bash[77809]: 7eabaaf7305d5ac2036bd05d074e84b2944a040542f605a8d0873cfeb7dc073e 2024-02-14T17:46:44.405 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 17:46:44 smithi100 systemd[1]: Started Ceph node-exporter.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:46:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:45.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:45.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:45.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:45 smithi179 ceph-mon[33633]: Deploying daemon node-exporter.b on smithi179 2024-02-14T17:46:45.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:45 smithi179 ceph-mon[33633]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:45.572 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:45 smithi179 bash[66606]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-14T17:46:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:45 smithi100 ceph-mon[30138]: Deploying daemon node-exporter.b on smithi179 2024-02-14T17:46:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:45 smithi100 ceph-mon[30138]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:45.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:45.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:45.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:45.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:45 smithi100 ceph-mon[39557]: Deploying daemon node-exporter.b on smithi179 2024-02-14T17:46:45.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:45 smithi100 ceph-mon[39557]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:46.321 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:45 smithi179 bash[66606]: Getting image source signatures 2024-02-14T17:46:46.321 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:45 smithi179 bash[66606]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-14T17:46:46.321 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:45 smithi179 bash[66606]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-14T17:46:46.321 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:45 smithi179 bash[66606]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-14T17:46:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:47 smithi179 ceph-mon[33633]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:47.571 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:47 smithi179 bash[66606]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-14T17:46:47.572 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:47 smithi179 bash[66606]: Writing manifest to image destination 2024-02-14T17:46:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:47 smithi100 ceph-mon[30138]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:47 smithi100 ceph-mon[39557]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:48.046 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:47 smithi179 podman[66606]: 2024-02-14 17:46:47.657025289 +0000 UTC m=+2.360914657 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-14T17:46:48.047 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:47 smithi179 podman[66606]: 2024-02-14 17:46:47.939854378 +0000 UTC m=+2.643743787 container create 1c0b8ee2fef23ef74f79d72c54a060004eb96a1340283f4b90203d469f86ac38 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T17:46:48.321 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 podman[66606]: 2024-02-14 17:46:48.159154208 +0000 UTC m=+2.863043592 container init 1c0b8ee2fef23ef74f79d72c54a060004eb96a1340283f4b90203d469f86ac38 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T17:46:48.322 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 podman[66606]: 2024-02-14 17:46:48.16581105 +0000 UTC m=+2.869700418 container start 1c0b8ee2fef23ef74f79d72c54a060004eb96a1340283f4b90203d469f86ac38 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T17:46:48.322 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.170Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-14T17:46:48.322 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.170Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-14T17:46:48.322 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.170Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-14T17:46:48.322 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.170Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-14T17:46:48.322 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.170Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-14T17:46:48.322 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.170Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-14T17:46:48.322 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-14T17:46:48.322 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=arp 2024-02-14T17:46:48.323 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-14T17:46:48.323 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-14T17:46:48.323 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-14T17:46:48.323 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-14T17:46:48.323 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-14T17:46:48.323 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-14T17:46:48.323 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=edac 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-14T17:46:48.324 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=os 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=stat 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-14T17:46:48.325 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=time 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=uname 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b[66658]: ts=2024-02-14T17:46:48.171Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 bash[66606]: 1c0b8ee2fef23ef74f79d72c54a060004eb96a1340283f4b90203d469f86ac38 2024-02-14T17:46:48.326 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 17:46:48 smithi179 systemd[1]: Started Ceph node-exporter.b for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:46:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:49 smithi179 ceph-mon[33633]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:49 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:49 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:49 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:49 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:49 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:49 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:46:49.587 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[39557]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:49.587 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.587 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.587 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.588 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:46:49.588 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[30138]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:49.588 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.588 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.588 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.588 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.588 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:49.588 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:49 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:46:50.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:46:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:46:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:46:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:51 smithi179 ceph-mon[33633]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:51 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:51 smithi100 ceph-mon[39557]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:51 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:51.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:51 smithi100 ceph-mon[30138]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:51.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:51 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:51.872 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:46:51 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:46:53.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:52 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:52 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:52 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:53.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:52 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:53.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:52 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:53.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:52 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:52 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:46:53.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:52 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:53.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:52 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:53 smithi179 ceph-mon[33633]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:53 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:53 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:53 smithi100 ceph-mon[39557]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:53 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:53 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:53 smithi100 ceph-mon[30138]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:53 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:53 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:55.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:54 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:46:55.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:54 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:46:55.117 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:54 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:55.117 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:54 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:46:55.118 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:54 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:46:55.118 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:54 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:54 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:46:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:54 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:46:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:54 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:46:56.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:55 smithi100 ceph-mon[30138]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:56.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:55 smithi100 ceph-mon[30138]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-14T17:46:56.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:55 smithi100 ceph-mon[30138]: Reconfiguring daemon alertmanager.a on smithi100 2024-02-14T17:46:56.132 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:55 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:56.133 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:55 smithi100 ceph-mon[39557]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:56.133 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:55 smithi100 ceph-mon[39557]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-14T17:46:56.133 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:55 smithi100 ceph-mon[39557]: Reconfiguring daemon alertmanager.a on smithi100 2024-02-14T17:46:56.133 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:55 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:56.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:55 smithi179 ceph-mon[33633]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:46:56.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:55 smithi179 ceph-mon[33633]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-14T17:46:56.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:55 smithi179 ceph-mon[33633]: Reconfiguring daemon alertmanager.a on smithi100 2024-02-14T17:46:56.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:55 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:46:58.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:57 smithi100 ceph-mon[30138]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:58.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:57 smithi100 ceph-mon[39557]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:58.151 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:57 smithi100 systemd[1]: Stopping Ceph alertmanager.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:46:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:57 smithi179 ceph-mon[33633]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:46:58.512 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:58 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[77165]: ts=2024-02-14T17:46:58.171Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-14T17:46:58.513 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:58 smithi100 podman[78675]: 2024-02-14 17:46:58.192046882 +0000 UTC m=+0.151783761 container died f0606bd736060bfb2ce30145098226e4546b772a4561ba3df5f0d7b86dfbb550 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:46:58.900 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:58 smithi100 podman[78675]: 2024-02-14 17:46:58.711368305 +0000 UTC m=+0.671105224 container cleanup f0606bd736060bfb2ce30145098226e4546b772a4561ba3df5f0d7b86dfbb550 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:46:58.900 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:58 smithi100 bash[78675]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a 2024-02-14T17:46:59.264 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:58 smithi100 podman[78694]: 2024-02-14 17:46:58.946463363 +0000 UTC m=+0.756678667 container remove f0606bd736060bfb2ce30145098226e4546b772a4561ba3df5f0d7b86dfbb550 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:46:59.264 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 podman[78694]: 2024-02-14 17:46:59.047068954 +0000 UTC m=+0.857284212 volume remove 898b2ca41ae229efd852ee806d5668af86bcb82284ac7505f33e2faf4a062fb8 2024-02-14T17:46:59.591 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-14T17:46:59.591 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 systemd[1]: Stopped Ceph alertmanager.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:46:59.591 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 systemd[1]: Starting Ceph alertmanager.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:46:59.591 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 podman[78794]: 2024-02-14 17:46:59.398531644 +0000 UTC m=+0.018543617 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-02-14T17:46:59.591 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 podman[78794]: 2024-02-14 17:46:59.506746023 +0000 UTC m=+0.126757996 volume create 87e963d41ee93f6ba3b90b3820ebeb62639fcd52dc8e88c39a913b1d7b6cb865 2024-02-14T17:46:59.591 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 podman[78794]: 2024-02-14 17:46:59.590557301 +0000 UTC m=+0.210569320 container create c2cd92038c03fe377d2a5553355dbfdfdffb83f94314ca6cfbf39bc2738a8c88 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:47:00.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:46:59 smithi179 ceph-mon[33633]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:00.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:46:59 smithi100 ceph-mon[30138]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:00.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:46:59 smithi100 ceph-mon[39557]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:00.401 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 podman[78794]: 2024-02-14 17:46:59.901456309 +0000 UTC m=+0.521468286 container init c2cd92038c03fe377d2a5553355dbfdfdffb83f94314ca6cfbf39bc2738a8c88 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:47:00.401 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 podman[78794]: 2024-02-14 17:46:59.90657601 +0000 UTC m=+0.526587986 container start c2cd92038c03fe377d2a5553355dbfdfdffb83f94314ca6cfbf39bc2738a8c88 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T17:47:00.401 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T17:46:59.919Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-14T17:47:00.401 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T17:46:59.919Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-14T17:47:00.401 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T17:46:59.920Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.100 port=9094 2024-02-14T17:47:00.401 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T17:46:59.920Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-14T17:47:00.401 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T17:46:59.943Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-14T17:47:00.401 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T17:46:59.944Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-14T17:47:00.402 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T17:46:59.946Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-14T17:47:00.402 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:46:59 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T17:46:59.946Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-14T17:47:00.402 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:47:00 smithi100 bash[78794]: c2cd92038c03fe377d2a5553355dbfdfdffb83f94314ca6cfbf39bc2738a8c88 2024-02-14T17:47:00.402 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:47:00 smithi100 systemd[1]: Started Ceph alertmanager.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:47:00.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:47:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:01.092 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:01 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:01.092 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:01 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:01.092 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:01 smithi179 ceph-mon[33633]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-14T17:47:01.092 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:01 smithi179 ceph-mon[33633]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:01.092 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:01 smithi179 ceph-mon[33633]: Reconfiguring daemon prometheus.a on smithi179 2024-02-14T17:47:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-mon[39557]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-14T17:47:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-mon[39557]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:01.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-mon[39557]: Reconfiguring daemon prometheus.a on smithi179 2024-02-14T17:47:01.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:01.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:01.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-mon[30138]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-14T17:47:01.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-mon[30138]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:01.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-mon[30138]: Reconfiguring daemon prometheus.a on smithi179 2024-02-14T17:47:01.893 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:47:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:47:02.175 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:02 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:02.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:02 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:02.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:02 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:02.401 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:47:01 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T17:47:01.921Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000893527s 2024-02-14T17:47:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-mon[33633]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:03 smithi100 ceph-mon[30138]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:03 smithi100 ceph-mon[39557]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:03.821 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 systemd[1]: Stopping Ceph prometheus.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:47:04.178 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.912Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.912Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.912Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.912Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.912Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.912Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.912Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.912Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.912Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.913Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.913Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[64336]: ts=2024-02-14T17:47:03.913Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-14T17:47:04.179 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:03 smithi179 podman[67702]: 2024-02-14 17:47:03.923246763 +0000 UTC m=+0.138772375 container died 0f6bf504887a0d3c5e4728bc6555cfc2833bc4c77c0502ce0e6b65b3b2d12bb6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:47:04.553 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:04 smithi179 podman[67702]: 2024-02-14 17:47:04.284754027 +0000 UTC m=+0.500279638 container cleanup 0f6bf504887a0d3c5e4728bc6555cfc2833bc4c77c0502ce0e6b65b3b2d12bb6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:47:04.553 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:04 smithi179 bash[67702]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a 2024-02-14T17:47:04.821 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:04 smithi179 podman[67721]: 2024-02-14 17:47:04.552350553 +0000 UTC m=+0.632097156 container remove 0f6bf504887a0d3c5e4728bc6555cfc2833bc4c77c0502ce0e6b65b3b2d12bb6 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:47:04.821 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:04 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-02-14T17:47:04.821 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:04 smithi179 systemd[1]: Stopped Ceph prometheus.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:47:04.822 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:04 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@prometheus.a.service: Consumed 1.169s CPU time. 2024-02-14T17:47:04.822 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:04 smithi179 systemd[1]: Starting Ceph prometheus.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T17:47:05.182 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:04 smithi179 podman[67823]: 2024-02-14 17:47:04.829006679 +0000 UTC m=+0.019692596 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-14T17:47:05.182 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:04 smithi179 podman[67823]: 2024-02-14 17:47:04.978542959 +0000 UTC m=+0.169228858 container create 75209456d1d62cbf131e00a319b7c4241a664d5365f9e54f9b11767fd1395a7b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:47:05.182 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 podman[67823]: 2024-02-14 17:47:05.181233472 +0000 UTC m=+0.371919371 container init 75209456d1d62cbf131e00a319b7c4241a664d5365f9e54f9b11767fd1395a7b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:47:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-mon[33633]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:05.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 podman[67823]: 2024-02-14 17:47:05.186557928 +0000 UTC m=+0.377243828 container start 75209456d1d62cbf131e00a319b7c4241a664d5365f9e54f9b11767fd1395a7b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T17:47:05.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.209Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-14T17:47:05.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.209Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-14T17:47:05.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.209Z caller=main.go:570 level=info host_details="(Linux 5.14.0-420.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Feb 9 18:10:28 UTC 2024 x86_64 smithi179 (none))" 2024-02-14T17:47:05.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.209Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-14T17:47:05.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.209Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-14T17:47:05.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.211Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-14T17:47:05.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.211Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-14T17:47:05.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.212Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-14T17:47:05.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.212Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.214Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.214Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.94µs 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.214Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.215Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.215Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.215Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=35.034µs wal_replay_duration=589.117µs wbl_replay_duration=88ns total_replay_duration=895.07µs 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.215Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.215Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.215Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.233Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=17.610388ms db_storage=704ns remote_storage=1.079µs web_handler=315ns query_engine=396ns scrape=446.59µs scrape_sd=67.99µs notify=9.072µs notify_sd=12.051µs rules=16.871124ms tracing=6.113µs 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.233Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T17:47:05.233Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 bash[67823]: 75209456d1d62cbf131e00a319b7c4241a664d5365f9e54f9b11767fd1395a7b 2024-02-14T17:47:05.573 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 17:47:05 smithi179 systemd[1]: Started Ceph prometheus.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T17:47:05.618 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-mon[39557]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-mon[30138]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STOPPING 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STOPPED 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STARTING 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Serving on http://:::9283 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STARTED 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STOPPING 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STOPPED 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STARTING 2024-02-14T17:47:05.619 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Serving on http://:::9283 2024-02-14T17:47:05.620 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STARTED 2024-02-14T17:47:05.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STOPPING 2024-02-14T17:47:05.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T17:47:05.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STOPPED 2024-02-14T17:47:05.901 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STARTING 2024-02-14T17:47:05.901 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Serving on http://:::9283 2024-02-14T17:47:05.901 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:05 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:17:47:05] ENGINE Bus STARTED 2024-02-14T17:47:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:47:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:47:06.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:47:06.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:47:06.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:47:06.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:06 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:47:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:47:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:47:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi100.front.sepia.ceph.com:9093"}]: dispatch 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi179.front.sepia.ceph.com:9095"}]: dispatch 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:47:06.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-14T17:47:06.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:47:06.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi179.front.sepia.ceph.com:3000"}]: dispatch 2024-02-14T17:47:06.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:06.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:06 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:47:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:07 smithi179 ceph-mon[33633]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:07.669 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:07 smithi100 ceph-mon[39557]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:07.669 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:07 smithi100 ceph-mon[30138]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:10.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:09 smithi179 ceph-mon[33633]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:10.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:09 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:09 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:09 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:09 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:09 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:47:10.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:09 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:47:10.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:09 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.150 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T17:47:09.924Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003079633s 2024-02-14T17:47:10.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[39557]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[30138]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:47:10.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:47:10.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:09 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:10.806 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:47:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:11.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:10 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:10 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:10 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:12.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:11 smithi179 ceph-mon[33633]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:12.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:47:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:47:12.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:11 smithi100 ceph-mon[39557]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:12.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:11 smithi100 ceph-mon[30138]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:13.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:12 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:12 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:13.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:12 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:14.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:13 smithi179 ceph-mon[33633]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:14.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:13 smithi100 ceph-mon[39557]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:14.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:13 smithi100 ceph-mon[30138]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:16.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:15 smithi179 ceph-mon[33633]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:16.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:15 smithi100 ceph-mon[39557]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:16.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:15 smithi100 ceph-mon[30138]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:18.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:17 smithi179 ceph-mon[33633]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:18.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:17 smithi100 ceph-mon[39557]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:18.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:17 smithi100 ceph-mon[30138]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:20.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:19 smithi179 ceph-mon[33633]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:20.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:19 smithi100 ceph-mon[30138]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:20.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:19 smithi100 ceph-mon[39557]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:20.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:47:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:22.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:21 smithi179 ceph-mon[33633]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:22.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:47:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:47:22.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:21 smithi100 ceph-mon[39557]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:21 smithi100 ceph-mon[30138]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:23.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:22 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:23.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:22 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:23.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:22 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:24.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:23 smithi100 ceph-mon[39557]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:24.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:23 smithi100 ceph-mon[30138]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:23 smithi179 ceph-mon[33633]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:26.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:25 smithi100 ceph-mon[39557]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:26.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:25 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:26.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:25 smithi100 ceph-mon[30138]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:26.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:25 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:25 smithi179 ceph-mon[33633]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:25 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:28.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:27 smithi100 ceph-mon[39557]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:27 smithi100 ceph-mon[30138]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:28.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:27 smithi179 ceph-mon[33633]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:29 smithi100 ceph-mon[30138]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:30.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:29 smithi100 ceph-mon[39557]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:30.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:29 smithi179 ceph-mon[33633]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:30.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:47:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:32.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:31 smithi179 ceph-mon[33633]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:32.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:47:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:47:32.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:31 smithi100 ceph-mon[39557]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:31 smithi100 ceph-mon[30138]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:32 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:32 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:32 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:34.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:33 smithi100 ceph-mon[30138]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:33 smithi100 ceph-mon[39557]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:33 smithi179 ceph-mon[33633]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:36.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:35 smithi100 ceph-mon[39557]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:36.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:35 smithi100 ceph-mon[30138]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:35 smithi179 ceph-mon[33633]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:38.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:37 smithi100 ceph-mon[30138]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:38.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:37 smithi100 ceph-mon[39557]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:38.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:37 smithi179 ceph-mon[33633]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:40.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:39 smithi100 ceph-mon[39557]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:40.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:39 smithi100 ceph-mon[30138]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:40.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:39 smithi179 ceph-mon[33633]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:40.867 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:40.867 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:47:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:40.868 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:41.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:42.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:41 smithi179 ceph-mon[33633]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:42.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:47:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:47:42.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:41 smithi100 ceph-mon[39557]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:42.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:41 smithi100 ceph-mon[30138]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:42 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:42 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:42 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:44.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:43 smithi100 ceph-mon[39557]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:44.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:43 smithi100 ceph-mon[30138]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:44.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:43 smithi179 ceph-mon[33633]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr:Note: switching to '8727ce939fc044a98f215e29696bb1433f5758d1'. 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr:state without impacting any branches by switching back to a branch. 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr: git switch -c 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr:Or undo this operation with: 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-02-14T17:47:44.922 INFO:tasks.workunit.client.0.smithi100.stderr: git switch - 2024-02-14T17:47:44.923 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-02-14T17:47:44.923 INFO:tasks.workunit.client.0.smithi100.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-14T17:47:44.923 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-02-14T17:47:44.923 INFO:tasks.workunit.client.0.smithi100.stderr:HEAD is now at 8727ce939fc Merge branch 'node-proxy-handle-fetch-oob-error' of https://github.com/guits/ceph into wip-guits-testing-2024-02-14-1033 2024-02-14T17:47:44.925 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-14T17:47:44.960 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-14T17:47:44.962 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-14T17:47:44.963 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-14T17:47:45.685 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-14T17:47:45.748 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-14T17:47:45.788 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-14T17:47:45.789 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-14T17:47:45.789 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-14T17:47:45.828 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-14T17:47:45.831 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-02-14T17:47:45.831 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-14T17:47:45.888 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-14T17:47:45.891 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-14T17:47:45.891 DEBUG:teuthology.orchestra.run.smithi100:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8727ce939fc044a98f215e29696bb1433f5758d1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-14T17:47:45.957 INFO:tasks.workunit.client.0.smithi100.stderr:+ parallel=1 2024-02-14T17:47:45.957 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' '' = --serial ']' 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ crimson=0 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' '' = --crimson ']' 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ color= 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -t 1 ']' 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ GTEST_OUTPUT=xml:/home/ubuntu/cephtest/archive/gtest_xml_report 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ declare -A pids 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_aio 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_aio' 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_aio 2024-02-14T17:47:45.958 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:45.959 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_aio 2024-02-14T17:47:45.959 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79525 2024-02-14T17:47:45.959 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_aio on pid 79525' 2024-02-14T17:47:45.959 INFO:tasks.workunit.client.0.smithi100.stdout:test api_aio on pid 79525 2024-02-14T17:47:45.962 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79525 2024-02-14T17:47:45.962 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:45.962 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:45.962 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-14T17:47:45.962 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_aio_pp 2024-02-14T17:47:45.962 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_aio_pp' 2024-02-14T17:47:45.962 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_aio_pp 2024-02-14T17:47:45.962 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:45.962 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.963 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:45.964 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:45.964 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:45.964 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.964 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:45.964 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.964 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:45.964 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:45.964 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_aio_pp 2024-02-14T17:47:45.964 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79532 2024-02-14T17:47:45.964 INFO:tasks.workunit.client.0.smithi100.stdout:test api_aio_pp on pid 79532 2024-02-14T17:47:45.968 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_aio_pp on pid 79532' 2024-02-14T17:47:45.968 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79532 2024-02-14T17:47:45.968 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:45.968 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:45.968 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-14T17:47:45.968 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_io 2024-02-14T17:47:45.968 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_io' 2024-02-14T17:47:45.968 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_io 2024-02-14T17:47:45.968 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.969 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:45.970 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:45.971 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_io 2024-02-14T17:47:45.972 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79541 2024-02-14T17:47:45.972 INFO:tasks.workunit.client.0.smithi100.stdout:test api_io on pid 79541 2024-02-14T17:47:45.978 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_io on pid 79541' 2024-02-14T17:47:45.978 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79541 2024-02-14T17:47:45.978 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:45.978 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-14T17:47:45.978 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:45.978 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_io_pp 2024-02-14T17:47:45.978 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_io_pp' 2024-02-14T17:47:45.978 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_io_pp 2024-02-14T17:47:45.978 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.979 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:45.980 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.981 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.982 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:45.982 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:45.982 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.982 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:45.982 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:45.982 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:45.982 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_io_pp 2024-02-14T17:47:45.982 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79552 2024-02-14T17:47:45.982 INFO:tasks.workunit.client.0.smithi100.stdout:test api_io_pp on pid 79552 2024-02-14T17:47:45.988 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_io_pp on pid 79552' 2024-02-14T17:47:45.988 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79552 2024-02-14T17:47:45.988 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-14T17:47:45.988 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:45.988 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:45.988 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_asio 2024-02-14T17:47:45.988 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_asio' 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_asio 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:45.989 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.990 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:45.990 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:45.990 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.990 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:45.990 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:45.990 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.990 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:45.990 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.990 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:45.991 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:45.992 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.993 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:45.994 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:45.995 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:45.995 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:45.995 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:45.995 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_asio 2024-02-14T17:47:45.995 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:45.995 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79570 2024-02-14T17:47:45.995 INFO:tasks.workunit.client.0.smithi100.stdout:test api_asio on pid 79570 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_asio on pid 79570' 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79570 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_list 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_list' 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_list 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.001 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.002 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.003 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.004 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.005 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.006 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79525/exe 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.007 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.008 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.009 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.009 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_list 2024-02-14T17:47:46.009 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79586 2024-02-14T17:47:46.009 INFO:tasks.workunit.client.0.smithi100.stdout:test api_list on pid 79586 2024-02-14T17:47:46.015 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_list on pid 79586' 2024-02-14T17:47:46.015 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79586 2024-02-14T17:47:46.015 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_lock 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_lock' 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.016 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.017 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_lock 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.018 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_aio: /' 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.019 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.020 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.020 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.020 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.020 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.020 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.020 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.020 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.020 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.020 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.021 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.021 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.021 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.021 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.021 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.021 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79532/exe 2024-02-14T17:47:46.021 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.022 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.022 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.022 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.022 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.022 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.022 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.022 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.022 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.022 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.023 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.024 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_lock 2024-02-14T17:47:46.025 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.025 INFO:tasks.workunit.client.0.smithi100.stdout:test api_lock on pid 79604 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79604 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_lock on pid 79604' 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79604 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-14T17:47:46.031 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_lock_pp 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_lock_pp' 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.032 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.033 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_lock_pp 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.034 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.035 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.036 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.037 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79541/exe 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.038 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.039 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.040 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.040 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-14T17:47:46.040 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.040 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.040 INFO:tasks.workunit.client.0.smithi100.stdout:test api_lock_pp on pid 79626 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_lock_pp 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79626 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_lock_pp on pid 79626' 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79626 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.046 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.047 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.047 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.047 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.047 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.047 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.047 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.047 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_io.log 2024-02-14T17:47:46.047 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.047 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.047 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.048 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.049 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_io: /' 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.050 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.051 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.052 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_misc 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_misc' 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79552/exe 2024-02-14T17:47:46.053 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_misc 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.054 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.055 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.056 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.057 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.058 INFO:tasks.workunit.client.0.smithi100.stdout:test api_misc on pid 79652 2024-02-14T17:47:46.063 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_misc 2024-02-14T17:47:46.064 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79652 2024-02-14T17:47:46.064 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_misc on pid 79652' 2024-02-14T17:47:46.064 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79652 2024-02-14T17:47:46.064 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.064 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.064 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-14T17:47:46.064 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.065 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.065 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.065 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.065 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.065 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.065 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.065 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.065 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.066 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.066 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.066 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.066 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.066 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.066 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.066 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.066 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.066 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.066 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_misc_pp 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_misc_pp' 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.067 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.068 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.068 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.068 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.068 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.068 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.068 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.068 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.068 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.068 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.068 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_misc_pp 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.069 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.070 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.070 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.070 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.070 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.070 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.070 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.070 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.071 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79586/exe 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79570/exe 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.072 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.073 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.073 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.073 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.073 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.073 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.073 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.073 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.074 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.074 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.074 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_misc_pp 2024-02-14T17:47:46.074 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.074 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79668 2024-02-14T17:47:46.074 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_misc_pp on pid 79668' 2024-02-14T17:47:46.074 INFO:tasks.workunit.client.0.smithi100.stdout:test api_misc_pp on pid 79668 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79668 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.079 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.080 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.081 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.082 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_list.log 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.083 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.084 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_list: /' 2024-02-14T17:47:46.084 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.084 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.084 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.084 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.084 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.084 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.084 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.085 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.085 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.085 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.085 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.085 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.085 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.085 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.085 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.086 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.087 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.087 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.087 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.087 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.087 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.087 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.087 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.087 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.087 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.087 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.088 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.088 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.088 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.088 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_asio: /' 2024-02-14T17:47:46.088 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.088 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.088 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.088 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.088 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.088 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.089 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.089 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.089 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.089 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.089 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.089 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.089 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.089 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.089 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.090 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79604/exe 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_tier_pp 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_tier_pp' 2024-02-14T17:47:46.091 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.092 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_lock: /' 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.093 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.094 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_tier_pp 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.095 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_tier_pp 2024-02-14T17:47:46.096 INFO:tasks.workunit.client.0.smithi100.stdout:test api_tier_pp on pid 79703 2024-02-14T17:47:46.099 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79703 2024-02-14T17:47:46.099 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_tier_pp on pid 79703' 2024-02-14T17:47:46.100 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79703 2024-02-14T17:47:46.100 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.100 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.100 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-14T17:47:46.100 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.100 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.100 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.100 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.100 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.100 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2024-02-14T17:47:46.101 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.101 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.101 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.101 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.101 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.101 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.101 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.101 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.101 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.102 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.102 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.102 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.102 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.102 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.102 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.102 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.102 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.102 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.103 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.103 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.103 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.103 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.103 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.103 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.103 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.103 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.103 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.104 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.104 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.104 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.104 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.104 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.104 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_pool 2024-02-14T17:47:46.104 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.104 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.104 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.105 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.105 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.105 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.105 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.105 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.105 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.105 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.105 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_pool' 2024-02-14T17:47:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.106 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.107 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.107 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.107 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.107 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.107 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.107 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.107 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.107 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.108 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.108 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.108 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.108 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_pool 2024-02-14T17:47:46.108 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.108 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.108 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.108 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.109 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.109 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.109 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.109 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.109 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.109 INFO:tasks.workunit.client.0.smithi100.stdout:test api_pool on pid 79729 2024-02-14T17:47:46.111 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_pool 2024-02-14T17:47:46.111 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79729 2024-02-14T17:47:46.112 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_pool on pid 79729' 2024-02-14T17:47:46.112 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79729 2024-02-14T17:47:46.112 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.112 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.112 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_snapshots 2024-02-14T17:47:46.112 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_snapshots' 2024-02-14T17:47:46.112 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.112 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.113 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.113 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.113 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.113 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.113 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.113 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.113 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.113 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.114 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.114 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.114 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.114 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.114 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.114 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.114 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.114 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.114 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.114 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_snapshots 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79652/exe 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.115 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79668/exe 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79626/exe 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.116 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.117 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_snapshots 2024-02-14T17:47:46.117 INFO:tasks.workunit.client.0.smithi100.stdout:test api_snapshots on pid 79742 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79742 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_snapshots on pid 79742' 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79742 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.121 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.122 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_snapshots_pp 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_snapshots_pp' 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.123 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.124 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.124 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_misc: /' 2024-02-14T17:47:46.124 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.124 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.124 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.124 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.124 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.124 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.124 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.125 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.125 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.125 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.125 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.125 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.125 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_snapshots_pp 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-14T17:47:46.126 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-14T17:47:46.127 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-14T17:47:46.127 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.127 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.127 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.127 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.127 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.127 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.127 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.127 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.128 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.128 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.128 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.128 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.128 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.128 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.128 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.128 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.128 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.129 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.129 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.129 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.129 INFO:tasks.workunit.client.0.smithi100.stdout:test api_snapshots_pp on pid 79776 2024-02-14T17:47:46.134 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_snapshots_pp 2024-02-14T17:47:46.134 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79776 2024-02-14T17:47:46.134 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_snapshots_pp on pid 79776' 2024-02-14T17:47:46.134 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79776 2024-02-14T17:47:46.134 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.134 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.134 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.134 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.134 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.135 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_stat 2024-02-14T17:47:46.136 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.137 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.138 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_stat' 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.139 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.140 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.141 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.142 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.142 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.142 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.142 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.142 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.142 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.142 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_stat 2024-02-14T17:47:46.142 INFO:tasks.workunit.client.0.smithi100.stdout:test api_stat on pid 79821 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_stat 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79821 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_stat on pid 79821' 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79821 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.149 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.150 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.150 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.150 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.150 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.150 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.150 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.150 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.151 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.151 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.151 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.151 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.151 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.151 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.152 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.152 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.152 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.152 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.152 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.152 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.152 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.152 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.153 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79703/exe 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.154 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_stat_pp 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_stat_pp' 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.155 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.156 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_stat_pp 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.157 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.158 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.159 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.160 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.161 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.162 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.163 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.164 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.164 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.164 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_stat_pp 2024-02-14T17:47:46.164 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.164 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.164 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.164 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.164 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79884 2024-02-14T17:47:46.164 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_stat_pp on pid 79884' 2024-02-14T17:47:46.164 INFO:tasks.workunit.client.0.smithi100.stdout:test api_stat_pp on pid 79884 2024-02-14T17:47:46.170 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.170 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79884 2024-02-14T17:47:46.170 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.170 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.170 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_watch_notify 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_watch_notify' 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-14T17:47:46.171 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.172 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.173 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79742/exe 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.174 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_watch_notify 2024-02-14T17:47:46.175 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.176 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.177 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.177 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.177 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.177 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.177 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.177 INFO:tasks.workunit.client.0.smithi100.stdout:test api_watch_notify on pid 79923 2024-02-14T17:47:46.183 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.183 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.183 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.183 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_watch_notify 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79923 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_watch_notify on pid 79923' 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79923 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.184 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.185 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.186 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-14T17:47:46.187 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.188 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.188 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.188 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.188 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.188 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_watch_notify_pp 2024-02-14T17:47:46.188 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_watch_notify_pp' 2024-02-14T17:47:46.188 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-14T17:47:46.188 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.188 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_watch_notify_pp 2024-02-14T17:47:46.188 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.189 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79821/exe 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.190 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.191 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.192 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.192 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.192 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.192 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.192 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.192 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.192 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.192 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.192 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.192 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.193 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.194 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.195 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79729/exe 2024-02-14T17:47:46.196 INFO:tasks.workunit.client.0.smithi100.stdout:test api_watch_notify_pp on pid 79957 2024-02-14T17:47:46.199 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_watch_notify_pp 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79957 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_watch_notify_pp on pid 79957' 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79957 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_stat: /' 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.200 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.201 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.202 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.203 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_cmd 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_cmd' 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.204 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_cmd 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.205 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.206 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.207 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.208 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.209 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.209 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.209 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.209 INFO:tasks.workunit.client.0.smithi100.stdout:test api_cmd on pid 79983 2024-02-14T17:47:46.212 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_cmd 2024-02-14T17:47:46.212 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79983 2024-02-14T17:47:46.212 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_cmd on pid 79983' 2024-02-14T17:47:46.212 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=79983 2024-02-14T17:47:46.212 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.212 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.212 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_pool: /' 2024-02-14T17:47:46.212 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-14T17:47:46.212 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_cmd_pp 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_cmd_pp' 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79776/exe 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.213 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.214 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.215 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.216 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.217 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2024-02-14T17:47:46.218 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.218 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_cmd_pp 2024-02-14T17:47:46.218 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.218 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.218 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.218 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.218 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.218 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-14T17:47:46.218 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_cmd_pp 2024-02-14T17:47:46.218 INFO:tasks.workunit.client.0.smithi100.stdout:test api_cmd_pp on pid 80008 2024-02-14T17:47:46.220 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80008 2024-02-14T17:47:46.220 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_cmd_pp on pid 80008' 2024-02-14T17:47:46.220 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80008 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_service 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_service' 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.221 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.222 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_service 2024-02-14T17:47:46.223 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.223 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.223 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.223 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.223 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.223 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.223 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.224 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.225 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.226 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.227 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.227 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.227 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.227 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.227 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.227 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.227 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.227 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.227 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.228 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.228 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.228 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.228 INFO:tasks.workunit.client.0.smithi100.stdout:test api_service on pid 80025 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_service 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80025 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_service on pid 80025' 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80025 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_service_pp 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79923/exe 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_service_pp' 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.231 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.232 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.233 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_service_pp 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.234 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-14T17:47:46.235 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2024-02-14T17:47:46.236 INFO:tasks.workunit.client.0.smithi100.stdout:test api_service_pp on pid 80038 2024-02-14T17:47:46.237 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_service_pp 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80038 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_service_pp on pid 80038' 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80038 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_c_write_operations 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_c_write_operations' 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.238 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_c_write_operations 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.239 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.240 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.240 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.240 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.240 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.240 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.240 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.240 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.240 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_c_write_operations 2024-02-14T17:47:46.240 INFO:tasks.workunit.client.0.smithi100.stdout:test api_c_write_operations on pid 80052 2024-02-14T17:47:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80052 2024-02-14T17:47:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_c_write_operations on pid 80052' 2024-02-14T17:47:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80052 2024-02-14T17:47:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.247 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.248 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.249 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.250 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.251 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.252 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.253 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.253 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.253 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.253 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.253 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.253 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.253 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.254 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.254 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.254 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.254 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.254 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.254 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.255 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.256 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.256 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.256 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.256 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.256 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.257 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.258 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79983/exe 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.259 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.260 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.261 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80008/exe 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_c_read_operations 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.262 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_c_read_operations' 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.263 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.264 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_cmd: /' 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79884/exe 2024-02-14T17:47:46.265 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.266 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.266 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.266 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.266 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.266 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.266 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.266 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.266 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.266 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.267 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.268 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-14T17:47:46.269 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2024-02-14T17:47:46.270 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:45 smithi100 ceph-mon[39557]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:46.270 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_c_read_operations 2024-02-14T17:47:46.270 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:45 smithi100 ceph-mon[30138]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:46.272 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.272 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.272 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.272 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.273 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.274 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.275 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.276 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.276 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.276 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.276 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.276 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.276 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.276 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.277 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.278 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/79957/exe 2024-02-14T17:47:46.279 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.280 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.280 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_c_read_operations 2024-02-14T17:47:46.280 INFO:tasks.workunit.client.0.smithi100.stdout:test api_c_read_operations on pid 80158 2024-02-14T17:47:46.284 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80158 2024-02-14T17:47:46.284 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_c_read_operations on pid 80158' 2024-02-14T17:47:46.285 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80158 2024-02-14T17:47:46.285 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.285 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.285 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-14T17:47:46.285 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.285 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.285 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.286 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.286 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.286 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.286 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.286 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.286 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.286 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.287 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.287 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.287 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80038/exe 2024-02-14T17:47:46.287 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.287 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.287 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.288 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.288 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.288 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.288 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.288 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.288 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.289 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.289 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.289 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2024-02-14T17:47:46.289 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s api_cls_remote_reads 2024-02-14T17:47:46.289 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' api_cls_remote_reads' 2024-02-14T17:47:46.289 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.290 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.290 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.290 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.290 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.290 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.290 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.290 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.291 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.291 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.291 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.291 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.291 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.291 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.292 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.292 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.292 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.292 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.292 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.292 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.293 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.293 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.293 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.293 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.293 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.293 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.294 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.294 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.294 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.294 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.294 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.294 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.294 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.294 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.295 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.295 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.295 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.295 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.295 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.295 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.295 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.296 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.296 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.296 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.296 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.296 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.297 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.297 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.297 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.297 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.297 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.297 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.297 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.298 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.298 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.298 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.298 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.298 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.299 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.299 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.299 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.299 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.299 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.299 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.300 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.300 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.300 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.300 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.300 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.300 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.301 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2024-02-14T17:47:46.301 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-14T17:47:46.301 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-14T17:47:46.301 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-14T17:47:46.301 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-14T17:47:46.301 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80025/exe 2024-02-14T17:47:46.301 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.302 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.302 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.302 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.302 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.302 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.302 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.303 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.303 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.303 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.303 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.303 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.303 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.304 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.304 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.304 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.304 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.304 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.304 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.305 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.305 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.305 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.305 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.305 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.305 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.305 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.306 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.306 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.306 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.306 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.306 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.306 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.306 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2024-02-14T17:47:46.307 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.307 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.307 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.307 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.307 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.307 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.307 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.308 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.308 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.308 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo api_cls_remote_reads 2024-02-14T17:47:46.308 INFO:tasks.workunit.client.0.smithi100.stdout:test api_cls_remote_reads on pid 80197 2024-02-14T17:47:46.313 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=api_cls_remote_reads 2024-02-14T17:47:46.313 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80197 2024-02-14T17:47:46.313 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test api_cls_remote_reads on pid 80197' 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80197 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.314 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.315 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s list_parallel 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' list_parallel' 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_service.log 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_service: /' 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.316 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.317 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.318 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo list_parallel 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.319 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.320 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.321 INFO:tasks.workunit.client.0.smithi100.stdout:test list_parallel on pid 80230 2024-02-14T17:47:46.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:45 smithi179 ceph-mon[33633]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:46.325 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.325 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.325 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.325 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.325 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.325 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.326 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=list_parallel 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80230 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test list_parallel on pid 80230' 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80230 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s open_pools_parallel 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' open_pools_parallel' 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.327 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80052/exe 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.328 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.329 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.330 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.331 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.332 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo open_pools_parallel 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.333 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.334 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.335 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.335 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.335 INFO:tasks.workunit.client.0.smithi100.stdout:test open_pools_parallel on pid 80278 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=open_pools_parallel 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80278 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80158/exe 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test open_pools_parallel on pid 80278' 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80278 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s delete_pools_parallel 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' delete_pools_parallel' 2024-02-14T17:47:46.337 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.338 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.339 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.340 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80197/exe 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.341 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo delete_pools_parallel 2024-02-14T17:47:46.342 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-02-14T17:47:46.342 INFO:tasks.workunit.client.0.smithi100.stdout:test delete_pools_parallel on pid 80308 2024-02-14T17:47:46.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=delete_pools_parallel 2024-02-14T17:47:46.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80308 2024-02-14T17:47:46.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test delete_pools_parallel on pid 80308' 2024-02-14T17:47:46.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80308 2024-02-14T17:47:46.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-02-14T17:47:46.344 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.344 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.345 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.346 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s cls 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-14T17:47:46.347 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' cls' 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.348 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.349 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.350 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.351 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.351 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.351 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.351 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo cls 2024-02-14T17:47:46.351 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.351 INFO:tasks.workunit.client.0.smithi100.stdout:test cls on pid 80322 2024-02-14T17:47:46.352 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=cls 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80322 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test cls on pid 80322' 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80322 2024-02-14T17:47:46.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s cmd 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' cmd' 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.354 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo cmd 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.355 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.356 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.357 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.357 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-14T17:47:46.357 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.357 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.357 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.357 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.357 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.357 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=cmd 2024-02-14T17:47:46.357 INFO:tasks.workunit.client.0.smithi100.stdout:test cmd on pid 80338 2024-02-14T17:47:46.359 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80338 2024-02-14T17:47:46.359 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test cmd on pid 80338' 2024-02-14T17:47:46.359 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80338 2024-02-14T17:47:46.359 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.359 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.359 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s handler_error 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' handler_error' 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo handler_error 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.360 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.361 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.362 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.362 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.362 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.362 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.362 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=handler_error 2024-02-14T17:47:46.362 INFO:tasks.workunit.client.0.smithi100.stdout:test handler_error on pid 80365 2024-02-14T17:47:46.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80365 2024-02-14T17:47:46.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test handler_error on pid 80365' 2024-02-14T17:47:46.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80365 2024-02-14T17:47:46.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.368 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.368 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.368 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.368 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.369 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.370 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.371 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80308/exe 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.372 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s io 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' io' 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.373 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.374 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.375 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.376 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.377 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.378 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.379 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.380 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.381 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo io 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.382 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.383 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2024-02-14T17:47:46.384 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.385 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80278/exe 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.386 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.387 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.388 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.389 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.390 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.390 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.390 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2024-02-14T17:47:46.390 INFO:tasks.workunit.client.0.smithi100.stdout:test io on pid 80449 2024-02-14T17:47:46.393 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=io 2024-02-14T17:47:46.393 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80449 2024-02-14T17:47:46.393 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test io on pid 80449' 2024-02-14T17:47:46.393 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80449 2024-02-14T17:47:46.393 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.393 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.393 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.394 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80230/exe 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.395 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s ec_io 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.396 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' ec_io' 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.399 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.400 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.401 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.401 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.401 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo ec_io 2024-02-14T17:47:46.401 INFO:tasks.workunit.client.0.smithi100.stdout:test ec_io on pid 80480 2024-02-14T17:47:46.404 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=ec_io 2024-02-14T17:47:46.404 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80480 2024-02-14T17:47:46.404 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test ec_io on pid 80480' 2024-02-14T17:47:46.404 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80480 2024-02-14T17:47:46.404 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.404 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.405 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s list 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' list' 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.406 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.407 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ list_parallel: /' 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.408 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-14T17:47:46.409 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo list 2024-02-14T17:47:46.410 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.411 INFO:tasks.workunit.client.0.smithi100.stdout:test list on pid 80497 2024-02-14T17:47:46.414 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=list 2024-02-14T17:47:46.414 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80497 2024-02-14T17:47:46.414 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test list on pid 80497' 2024-02-14T17:47:46.414 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80497 2024-02-14T17:47:46.414 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.414 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.414 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s ec_list 2024-02-14T17:47:46.414 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' ec_list' 2024-02-14T17:47:46.414 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.414 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.415 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80322/exe 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.416 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.417 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.417 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.417 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.417 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.417 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.417 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.417 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.417 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.418 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.418 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.418 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.418 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.418 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.418 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.418 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.418 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.419 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.419 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.419 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.419 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.419 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.419 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.419 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.419 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.419 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.420 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.420 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.420 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.420 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.420 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_cls 2024-02-14T17:47:46.420 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_cls.log 2024-02-14T17:47:46.420 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo ec_list 2024-02-14T17:47:46.420 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.421 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.421 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.421 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.421 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.421 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.421 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.421 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.422 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.422 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.422 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.422 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.422 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.422 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.422 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.422 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.423 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.423 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.423 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.423 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.423 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.423 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.423 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ cls: /' 2024-02-14T17:47:46.423 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.423 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.424 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.424 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.424 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.424 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.424 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.424 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.424 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.424 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.425 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.425 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.425 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.425 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.425 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.425 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.425 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.425 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.426 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.426 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.426 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.426 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.426 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.426 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.427 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80338/exe 2024-02-14T17:47:46.427 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=ec_list 2024-02-14T17:47:46.427 INFO:tasks.workunit.client.0.smithi100.stdout:test ec_list on pid 80545 2024-02-14T17:47:46.434 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80545 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test ec_list on pid 80545' 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80545 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.435 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.436 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.437 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.437 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s misc 2024-02-14T17:47:46.437 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' misc' 2024-02-14T17:47:46.437 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.437 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.437 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.437 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.437 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.437 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.438 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.438 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.438 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.438 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.438 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.438 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.438 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.439 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.439 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.439 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.439 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.439 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.439 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.439 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.439 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.440 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.440 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.440 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.440 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.440 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.440 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.441 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_cmd 2024-02-14T17:47:46.441 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.442 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.442 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.442 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.442 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.442 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ cmd: /' 2024-02-14T17:47:46.442 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80449/exe 2024-02-14T17:47:46.442 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.443 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.443 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.443 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.443 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.443 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.443 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.443 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.444 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.444 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.444 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.444 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.444 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.444 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.445 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.445 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.445 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.445 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.445 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.445 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.446 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.446 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.446 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.446 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.446 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.446 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.446 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.446 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_cmd.log 2024-02-14T17:47:46.446 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.447 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.447 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.447 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.447 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.447 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.447 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.448 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.448 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.448 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.448 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.448 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.448 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.449 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.449 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.449 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.449 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.449 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.449 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.449 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.450 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.450 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.450 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.450 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.450 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.450 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.450 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.451 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.451 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.451 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.451 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.451 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.451 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ io: /' 2024-02-14T17:47:46.452 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.452 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.452 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.452 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.452 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.452 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.453 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.453 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.453 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.453 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.453 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.453 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.454 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.454 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.454 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.454 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.454 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.454 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.454 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.455 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.455 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.455 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.455 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.455 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.455 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.456 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.456 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_io.log 2024-02-14T17:47:46.456 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.456 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.456 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.456 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.456 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.457 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.457 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.457 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.457 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.457 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.457 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.458 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.458 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.458 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.458 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.458 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.458 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.459 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.459 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.459 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.459 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.459 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.459 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.459 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.459 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.459 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.460 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.460 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.460 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.460 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.460 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.460 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.460 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_io 2024-02-14T17:47:46.460 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.461 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.461 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.461 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.461 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.461 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.461 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.461 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.461 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.461 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.461 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.462 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.462 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.462 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.462 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.462 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.462 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.462 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.462 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.463 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.463 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.463 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.463 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.463 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.463 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.463 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.463 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.464 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.465 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.466 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80497/exe 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.467 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80480/exe 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.468 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ list: /' 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_list 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_list.log 2024-02-14T17:47:46.469 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.470 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.470 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.470 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.470 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.470 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo misc 2024-02-14T17:47:46.470 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.470 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.470 INFO:tasks.workunit.client.0.smithi100.stdout:test misc on pid 80608 2024-02-14T17:47:46.473 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=misc 2024-02-14T17:47:46.473 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80608 2024-02-14T17:47:46.473 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test misc on pid 80608' 2024-02-14T17:47:46.473 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80608 2024-02-14T17:47:46.473 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.473 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s pool 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' pool' 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo pool 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.474 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80365/exe 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.475 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.476 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80545/exe 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_handler_error.log 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ handler_error: /' 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.477 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ ec_list: /' 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_handler_error 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_ec_list.log 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_ec_list 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_ec_io 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.478 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_ec_io.log 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ ec_io: /' 2024-02-14T17:47:46.479 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.480 INFO:tasks.workunit.client.0.smithi100.stdout:test pool on pid 80686 2024-02-14T17:47:46.481 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=pool 2024-02-14T17:47:46.481 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80686 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test pool on pid 80686' 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80686 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s read_operations 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' read_operations' 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.482 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo read_operations 2024-02-14T17:47:46.483 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.484 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.485 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.486 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.486 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.486 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.486 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.486 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.486 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.486 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.486 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.486 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.486 INFO:tasks.workunit.client.0.smithi100.stdout:test read_operations on pid 80752 2024-02-14T17:47:46.489 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=read_operations 2024-02-14T17:47:46.489 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80752 2024-02-14T17:47:46.489 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test read_operations on pid 80752' 2024-02-14T17:47:46.489 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80752 2024-02-14T17:47:46.489 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.489 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.489 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-02-14T17:47:46.489 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s snapshots 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' snapshots' 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo snapshots 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.490 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.491 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.492 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.493 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.494 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.494 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.494 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.494 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.494 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.494 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.494 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.494 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.494 INFO:tasks.workunit.client.0.smithi100.stdout:test snapshots on pid 80810 2024-02-14T17:47:46.495 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=snapshots 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80810 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test snapshots on pid 80810' 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80810 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s watch_notify 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' watch_notify' 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.496 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo watch_notify 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.497 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.498 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.502 INFO:tasks.workunit.client.0.smithi100.stdout:test watch_notify on pid 80827 2024-02-14T17:47:46.506 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=watch_notify 2024-02-14T17:47:46.506 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80608/exe 2024-02-14T17:47:46.506 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80827 2024-02-14T17:47:46.506 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test watch_notify on pid 80827' 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80827 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:++ printf %25s write_operations 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.507 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:+ r=' write_operations' 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.508 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:++ echo write_operations 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.509 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.510 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:++ awk '{print $1}' 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.511 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.512 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_misc.log 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_misc 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.513 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.514 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.514 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.514 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.514 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.514 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.514 INFO:tasks.workunit.client.0.smithi100.stderr:+ ff=write_operations 2024-02-14T17:47:46.514 INFO:tasks.workunit.client.0.smithi100.stdout:test write_operations on pid 80852 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80852 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo 'test write_operations on pid 80852' 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:+ pids[$f]=80852 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:+ ret=0 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 1 -eq 1 ']' 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79703 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79703 2024-02-14T17:47:46.527 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.528 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.528 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.528 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.528 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.528 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ misc: /' 2024-02-14T17:47:46.528 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.529 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.529 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.529 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.529 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.529 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.529 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.529 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.529 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.529 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.529 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.530 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.530 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.530 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.530 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.530 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.530 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.530 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.530 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.530 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.531 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.531 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.531 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.531 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.531 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.531 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.531 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.532 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.532 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.532 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.532 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.532 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.532 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.532 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.533 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.533 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -f /etc/bashrc ']' 2024-02-14T17:47:46.533 INFO:tasks.workunit.client.0.smithi100.stderr:+ . /etc/bashrc 2024-02-14T17:47:46.533 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.533 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -z '' ']' 2024-02-14T17:47:46.534 INFO:tasks.workunit.client.0.smithi100.stderr:++ BASHRCSOURCED=Y 2024-02-14T17:47:46.534 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.534 INFO:tasks.workunit.client.0.smithi100.stderr:++ shopt -q login_shell 2024-02-14T17:47:46.534 INFO:tasks.workunit.client.0.smithi100.stderr:+++ umask 2024-02-14T17:47:46.534 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.534 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.534 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.535 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.535 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.535 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.535 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.535 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.535 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.536 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.536 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.536 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.536 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.536 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.536 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.537 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.537 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.537 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.537 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.537 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.537 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.537 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.538 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.538 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.538 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.538 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.538 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.538 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.538 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.539 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.539 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.539 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.539 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' 0022 -eq 0 ']' 2024-02-14T17:47:46.539 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.539 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.539 INFO:tasks.workunit.client.0.smithi100.stderr:++ SHELL=/bin/bash 2024-02-14T17:47:46.540 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.540 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.540 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.540 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.540 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.540 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.541 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.541 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-14T17:47:46.541 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.541 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.541 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-14T17:47:46.541 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.542 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-14T17:47:46.542 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.542 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-14T17:47:46.542 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.542 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80752/exe 2024-02-14T17:47:46.542 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-14T17:47:46.542 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.543 INFO:tasks.workunit.client.0.smithi100.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-14T17:47:46.543 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.543 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80686/exe 2024-02-14T17:47:46.543 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-14T17:47:46.543 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.543 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-14T17:47:46.543 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.544 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.544 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.544 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.544 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.544 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.544 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.545 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.545 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.545 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.545 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.545 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.545 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.546 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.546 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.546 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.546 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.546 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.546 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.546 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.547 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.547 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.547 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.547 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.547 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.547 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.547 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.548 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.548 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.548 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.548 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.548 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.549 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.549 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.549 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.549 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.549 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.549 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.549 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.550 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.550 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.550 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.550 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.550 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.550 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.550 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.551 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.551 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.551 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.551 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.551 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.551 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.552 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.552 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.552 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.552 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_pool 2024-02-14T17:47:46.552 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_pool.log 2024-02-14T17:47:46.552 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.553 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ read_operations: /' 2024-02-14T17:47:46.553 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.553 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.553 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.553 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'grep=grep --color=auto' 2024-02-14T17:47:46.553 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.553 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.554 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.554 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-14T17:47:46.554 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.554 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-14T17:47:46.554 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_read_operations.log 2024-02-14T17:47:46.554 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.555 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-14T17:47:46.555 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.555 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.555 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorls.sh 2024-02-14T17:47:46.555 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.555 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.556 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.556 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.556 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' '!' -t 0 ']' 2024-02-14T17:47:46.556 INFO:tasks.workunit.client.0.smithi100.stderr:+++ return 2024-02-14T17:47:46.556 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.556 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-14T17:47:46.557 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.557 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-14T17:47:46.557 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export S_COLORS=auto 2024-02-14T17:47:46.557 INFO:tasks.workunit.client.0.smithi100.stderr:+++ S_COLORS=auto 2024-02-14T17:47:46.557 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.557 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-14T17:47:46.558 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_read_operations 2024-02-14T17:47:46.558 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.558 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-14T17:47:46.558 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.558 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.558 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.559 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.559 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.559 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.559 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ pool: /' 2024-02-14T17:47:46.559 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.559 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.560 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.560 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.560 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.560 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.560 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.560 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.561 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.561 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.561 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.561 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.561 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.561 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.561 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.562 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.562 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.562 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.562 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.562 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.562 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.563 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.563 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80810/exe 2024-02-14T17:47:46.563 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.563 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.563 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.563 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.563 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.564 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-14T17:47:46.564 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-14T17:47:46.564 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-14T17:47:46.564 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.564 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-14T17:47:46.565 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.565 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-14T17:47:46.565 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-14T17:47:46.565 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.565 INFO:tasks.workunit.client.0.smithi100.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-14T17:47:46.565 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.565 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.566 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.566 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.566 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.566 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.566 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.566 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.566 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.567 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.567 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.567 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.567 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.567 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.567 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.568 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.568 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.568 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.568 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.568 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_snapshots 2024-02-14T17:47:46.568 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.569 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.569 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.569 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.569 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.569 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.569 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.569 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.570 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.570 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.570 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.570 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.570 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.570 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_snapshots.log 2024-02-14T17:47:46.570 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80827/exe 2024-02-14T17:47:46.570 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-14T17:47:46.570 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-14T17:47:46.570 INFO:tasks.workunit.client.0.smithi100.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-14T17:47:46.571 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.571 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-14T17:47:46.571 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ snapshots: /' 2024-02-14T17:47:46.571 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.571 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-14T17:47:46.571 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-14T17:47:46.571 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.571 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-14T17:47:46.571 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.571 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/gawk.sh 2024-02-14T17:47:46.572 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.572 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-14T17:47:46.572 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.572 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/lang.sh 2024-02-14T17:47:46.572 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup 2024-02-14T17:47:46.572 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.572 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/locale 2024-02-14T17:47:46.572 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.572 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.573 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.573 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.573 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.573 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.573 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.573 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.573 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.573 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.573 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.574 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-14T17:47:46.574 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_watch_notify 2024-02-14T17:47:46.574 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.574 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-14T17:47:46.574 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-14T17:47:46.574 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ watch_notify: /' 2024-02-14T17:47:46.574 INFO:tasks.workunit.client.0.smithi100.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-14T17:47:46.574 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-02-14T17:47:46.574 INFO:tasks.workunit.client.0.smithi100.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:++++ export LANG=en_US.UTF-8 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:++++ LANG=en_US.UTF-8 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:+++ LANG=en_US.UTF-8 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:+++ unset LANG_backup config 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n '' ']' 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' dumb = linux ']' 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/less.sh 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:++ for i in /etc/profile.d/*.sh 2024-02-14T17:47:46.575 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:++ '[' '' ']' 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:++ . /etc/profile.d/which2.sh 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:+++++ readlink /proc/80852/exe 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:++++ basename /usr/bin/bash 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_declare='declare -f' 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:+++ which_opt=-f 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export which_declare 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:+++ export -f which 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset i 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:++ unset -f pathmunge 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:+ export PATH 2024-02-14T17:47:46.576 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-14T17:47:46.577 INFO:tasks.workunit.client.0.smithi100.stderr:+ unset rc 2024-02-14T17:47:46.577 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph_test_neorados_write_operations 2024-02-14T17:47:46.577 INFO:tasks.workunit.client.0.smithi100.stderr:+ sed 's/^/ write_operations: /' 2024-02-14T17:47:46.577 INFO:tasks.workunit.client.0.smithi100.stderr:+ tee ceph_test_neorados_write_operations.log 2024-02-14T17:47:47.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3141801077' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-14T17:47:47.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi100-80594-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3141801077' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-14T17:47:47.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi100-80594-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3141801077' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-14T17:47:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi100-80594-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi100-80625-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/862977918' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-79590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3190750646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-79645-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1215395011' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-79612-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1862164061' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-79691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:48 smithi179 ceph-mon[33633]: osdmap e73: 8 total, 8 up, 8 in 2024-02-14T17:47:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi100-80625-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/862977918' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-79590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3190750646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-79645-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1215395011' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-79612-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1862164061' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-79691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[30138]: osdmap e73: 8 total, 8 up, 8 in 2024-02-14T17:47:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi100-80625-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/862977918' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-79590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3190750646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-79645-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1215395011' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-79612-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1862164061' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-79691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:48 smithi100 ceph-mon[39557]: osdmap e73: 8 total, 8 up, 8 in 2024-02-14T17:47:50.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:47:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:47:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:47:52.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:47:51 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:47:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3827725440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-79925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1043584586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-79752-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1804549539' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-80125-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[30138]: osdmap e74: 8 total, 8 up, 8 in 2024-02-14T17:47:53.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi100-80594-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:53.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:53.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3827725440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-79925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1043584586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-79752-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1804549539' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-80125-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[39557]: osdmap e74: 8 total, 8 up, 8 in 2024-02-14T17:47:53.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi100-80594-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:53.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:53.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3827725440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-79925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1043584586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-79752-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1804549539' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-80125-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:53.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:53 smithi179 ceph-mon[33633]: osdmap e74: 8 total, 8 up, 8 in 2024-02-14T17:47:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi100-80594-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2386979394' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80097-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/515896951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80298-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1275917253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/348635410' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3322867081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-80034-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3808260238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: osdmap e75: 8 total, 8 up, 8 in 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi100-80625-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi100-80594-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3298038245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-79999-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3996527446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-79629-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4021617345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-80289-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1596912450' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80086-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi100-80594-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1681172585' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: osdmap e76: 8 total, 8 up, 8 in 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-79590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-79645-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-79612-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-79691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:54.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2386979394' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80097-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/515896951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80298-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1275917253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/348635410' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3322867081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-80034-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3808260238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: osdmap e75: 8 total, 8 up, 8 in 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi100-80625-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi100-80594-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3298038245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-79999-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3996527446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-79629-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4021617345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-80289-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1596912450' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80086-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi100-80594-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:47:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1681172585' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:47:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: osdmap e76: 8 total, 8 up, 8 in 2024-02-14T17:47:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2386979394' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80097-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/515896951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80298-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1275917253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/348635410' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3322867081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-80034-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3808260238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:54.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: osdmap e75: 8 total, 8 up, 8 in 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi100-80625-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi100-80594-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3298038245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-79999-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3996527446' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-79629-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4021617345' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-80289-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1596912450' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80086-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi100-80594-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1681172585' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: osdmap e76: 8 total, 8 up, 8 in 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-79590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-79645-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-79612-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-79691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-79590-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-79645-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-79612-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-79691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-14T17:47:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [----------] Global test environment set-up. 2024-02-14T17:47:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-14T17:47:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-14T17:47:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1 ms) 2024-02-14T17:47:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-14T17:47:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (0 ms) 2024-02-14T17:47:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-14T17:47:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-02-14T17:47:54.948 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (11 ms) 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (2 ms) 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (1 ms) 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-14T17:47:54.949 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (1 ms) 2024-02-14T17:47:54.950 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-14T17:47:54.950 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (1 ms) 2024-02-14T17:47:54.950 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-14T17:47:54.950 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (1 ms) 2024-02-14T17:47:54.950 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [----------] 12 tests from AsioRados (21 ms total) 2024-02-14T17:47:54.950 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: 2024-02-14T17:47:54.950 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [----------] Global test environment tear-down 2024-02-14T17:47:54.950 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (8885 ms total) 2024-02-14T17:47:54.950 INFO:tasks.workunit.client.0.smithi100.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: pgmap v68: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: pgmap v70: 420 pgs: 288 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: pgmap v71: 420 pgs: 220 creating+peering, 154 active+clean, 46 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: pgmap v74: 1124 pgs: 220 creating+peering, 154 active+clean, 750 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-79925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-80125-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-79752-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:55.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80097-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80298-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi100-80625-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-79590-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-79645-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-79612-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-79691-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-79925-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-80125-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-79752-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: osdmap e77: 8 total, 8 up, 8 in 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-80034-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-79999-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-79629-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi100-80625-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-79670-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-80289-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/27467051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-79838-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80086-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi100-80594-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:47:55.825 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:55.826 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-79670-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.826 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-79838-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.826 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi100-80625-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:47:55.826 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:55.826 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:55 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:55.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: pgmap v68: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: pgmap v70: 420 pgs: 288 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: pgmap v71: 420 pgs: 220 creating+peering, 154 active+clean, 46 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: pgmap v74: 1124 pgs: 220 creating+peering, 154 active+clean, 750 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-79925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-80125-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-79752-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80097-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80298-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi100-80625-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-79590-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-79645-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-79612-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-79691-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-79925-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-80125-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-79752-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: osdmap e77: 8 total, 8 up, 8 in 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-80034-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:55.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-79999-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-79629-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi100-80625-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-79670-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-80289-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/27467051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-79838-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80086-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi100-80594-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-79670-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-79838-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi100-80625-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:47:55.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:55.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:55.905 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: pgmap v68: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:47:55.905 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: pgmap v70: 420 pgs: 288 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T17:47:55.905 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:47:55.905 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: pgmap v71: 420 pgs: 220 creating+peering, 154 active+clean, 46 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:47:55.905 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: pgmap v74: 1124 pgs: 220 creating+peering, 154 active+clean, 750 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-79925-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-80125-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-79752-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80097-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80298-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi100-80625-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi100-79590-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi100-79645-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi100-79612-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi100-79691-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi100-79925-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi100-80125-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi100-79752-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: osdmap e77: 8 total, 8 up, 8 in 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-80034-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-79953-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-79999-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-79629-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi100-80625-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-79670-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-80289-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/27467051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-79838-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80086-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.907 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi100-80594-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:47:55.908 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:55.908 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-79670-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.908 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-79838-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:55.908 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi100-80625-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:47:55.908 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:47:55.908 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:55 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:47:56.413 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-02-14T17:47:56.413 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] Global test environment set-up. 2024-02-14T17:47:56.413 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-14T17:47:56.413 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-14T17:47:56.413 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.ListObjects (345 ms) 2024-02-14T17:47:56.413 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-14T17:47:56.413 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (34 ms) 2024-02-14T17:47:56.413 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo1,foo2,foo3 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo1 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo2 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo3 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo1,foo4,foo5 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo4 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo5 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo1 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo6,foo7 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo7 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo6 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo4 2024-02-14T17:47:56.414 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo5 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns2:foo7 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns2:foo6 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo1 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo1 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo2 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo3 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (84 ms) 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 1 0 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 10 0 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 13 0 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 7 0 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 14 0 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 0 0 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 15 0 2024-02-14T17:47:56.415 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 11 0 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 5 0 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 8 0 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 6 0 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 3 0 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 4 0 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 12 0 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 9 0 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 2 0 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (64 ms) 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: x cursor=MIN 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=1 cursor=38:02547ec2:::1:head 2024-02-14T17:47:56.416 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=10 cursor=38:52ea6a34:::10:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=13 cursor=38:566253c9:::13:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=7 cursor=38:5c6b0b28:::7:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=14 cursor=38:62a1935d:::14:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=0 cursor=38:6cac518f:::0:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=15 cursor=38:863748b0:::15:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=11 cursor=38:89d3ae78:::11:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=5 cursor=38:b29083e3:::5:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=8 cursor=38:bd63b0f1:::8:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=6 cursor=38:c4fdafeb:::6:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=3 cursor=38:cfc208b3:::3:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=4 cursor=38:d83876eb:::4:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=12 cursor=38:de5d7c5f:::12:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=9 cursor=38:e960b815:::9:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > oid=2 cursor=38:f905c69b:::2:head 2024-02-14T17:47:56.417 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=1 cursor=38:02547ec2:::1:head 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:02547ec2:::1:head 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:02547ec2:::1:head -> 1 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=10 cursor=38:52ea6a34:::10:head 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:52ea6a34:::10:head 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:52ea6a34:::10:head -> 10 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=13 cursor=38:566253c9:::13:head 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:566253c9:::13:head 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:566253c9:::13:head -> 13 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=7 cursor=38:5c6b0b28:::7:head 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:5c6b0b28:::7:head 2024-02-14T17:47:56.418 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:5c6b0b28:::7:head -> 7 2024-02-14T17:47:56.481 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=14 cursor=38:62a1935d:::14:head 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:62a1935d:::14:head 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:62a1935d:::14:head -> 14 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=0 cursor=38:6cac518f:::0:head 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:6cac518f:::0:head 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:6cac518f:::0:head -> 0 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=15 cursor=38:863748b0:::15:head 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:863748b0:::15:head 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:863748b0:::15:head -> 15 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=11 cursor=38:89d3ae78:::11:head 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:89d3ae78:::11:head 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:89d3ae78:::11:head -> 11 2024-02-14T17:47:56.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=5 cursor=38:b29083e3:::5:head 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:b29083e3:::5:head 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:b29083e3:::5:head -> 5 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=8 cursor=38:bd63b0f1:::8:head 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:bd63b0f1:::8:head 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:bd63b0f1:::8:head -> 8 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=6 cursor=38:c4fdafeb:::6:head 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:c4fdafeb:::6:head 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:c4fdafeb:::6:head -> 6 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=3 cursor=38:cfc208b3:::3:head 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:cfc208b3:::3:head 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:cfc208b3:::3:head -> 3 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=4 cursor=38:d83876eb:::4:head 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:d83876eb:::4:head 2024-02-14T17:47:56.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:d83876eb:::4:head -> 4 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=12 cursor=38:de5d7c5f:::12:head 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:de5d7c5f:::12:head 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:de5d7c5f:::12:head -> 12 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=9 cursor=38:e960b815:::9:head 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:e960b815:::9:head 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:e960b815:::9:head -> 9 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : oid=2 cursor=38:f905c69b:::2:head 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:f905c69b:::2:head 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:f905c69b:::2:head -> 2 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:f905c69b:::2:head 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:f905c69b:::2:head expected=38:f905c69b:::2:head 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:f905c69b:::2:head -> 2 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=2 expected=2 2024-02-14T17:47:56.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:e960b815:::9:head 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:e960b815:::9:head expected=38:e960b815:::9:head 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:e960b815:::9:head -> 9 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=9 expected=9 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:bd63b0f1:::8:head 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:bd63b0f1:::8:head expected=38:bd63b0f1:::8:head 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:bd63b0f1:::8:head -> 8 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=8 expected=8 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:b29083e3:::5:head 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:b29083e3:::5:head expected=38:b29083e3:::5:head 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:b29083e3:::5:head -> 5 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=5 expected=5 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:89d3ae78:::11:head 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:89d3ae78:::11:head expected=38:89d3ae78:::11:head 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:89d3ae78:::11:head -> 11 2024-02-14T17:47:56.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=11 expected=11 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:863748b0:::15:head 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:863748b0:::15:head expected=38:863748b0:::15:head 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:863748b0:::15:head -> 15 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=15 expected=15 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:6cac518f:::0:head 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:6cac518f:::0:head expected=38:6cac518f:::0:head 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:6cac518f:::0:head -> 0 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=0 expected=0 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:de5d7c5f:::12:head 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:de5d7c5f:::12:head expected=38:de5d7c5f:::12:head 2024-02-14T17:47:56.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:de5d7c5f:::12:head -> 12 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4241145333' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80097-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80298-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-80034-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-79953-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-79999-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-79629-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-80289-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80086-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:57.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-79670-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-79838-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-80161-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:57.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: osdmap e78: 8 total, 8 up, 8 in 2024-02-14T17:47:57.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-80161-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4241145333' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:47:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80097-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80298-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-80034-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-79953-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-79999-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-79629-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-80289-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80086-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-79670-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-79838-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-80161-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: osdmap e78: 8 total, 8 up, 8 in 2024-02-14T17:47:57.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-80161-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4241145333' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:47:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80097-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80298-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi100-80034-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi100-79953-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi100-79999-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi100-79629-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi100-80289-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80086-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:47:57.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi100-79670-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi100-79838-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:47:57.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-80161-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:57.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: osdmap e78: 8 total, 8 up, 8 in 2024-02-14T17:47:57.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-80161-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:47:57.369 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : ent list_parallel: process_1_[80509]: starting. 2024-02-14T17:47:57.369 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[80509]: creating pool ceph_test_rados_list_parallel.smithi100-80467 2024-02-14T17:47:57.369 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[80509]: created object 0... 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[80509]: created object 25... 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[80509]: created object 49... 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[80509]: finishing. 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_1_[80509]: shutting down. 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[80510]: starting. 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[80510]: listing objects. 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[80510]: listed object 0... 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[80510]: listed object 25... 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[80510]: saw 50 objects 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_2_[80510]: shutting down. 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[81592]: starting. 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[81592]: creating pool ceph_test_rados_list_parallel.smithi100-80467 2024-02-14T17:47:57.370 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[81592]: created object 0... 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[81592]: created object 25... 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[81592]: created object 49... 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[81592]: finishing. 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_3_[81592]: shutting down. 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[81593]: starting. 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[81593]: listing objects. 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[81593]: listed object 0... 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[81593]: listed object 25... 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[81593]: saw 43 objects 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_4_[81593]: shutting down. 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[81594]: starting. 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[81594]: removed 25 objects... 2024-02-14T17:47:57.371 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[81594]: removed half of the objects 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[81594]: removed 50 objects... 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[81594]: removed 50 objects 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_5_[81594]: shutting down. 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[81700]: starting. 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[81700]: creating pool ceph_test_rados_list_parallel.smithi100-80467 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[81700]: created object 0... 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[81700]: created object 25... 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[81700]: created object 49... 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[81700]: finishing. 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_6_[81700]: shutting down. 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.372 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[81701]: starting. 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[81701]: listing objects. 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[81701]: listed object 0... 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[81701]: listed object 25... 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[81701]: listed object 50... 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[81701]: saw 55 objects 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_7_[81701]: shutting down. 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[81702]: starting. 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[81702]: added 25 objects... 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[81702]: added half of the objects 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[81702]: added 50 objects... 2024-02-14T17:47:57.373 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[81702]: added 50 objects 2024-02-14T17:47:57.374 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_8_[81702]: shutting down. 2024-02-14T17:47:57.374 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.374 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.566 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.566 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[81748]: starting. 2024-02-14T17:47:57.566 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[81748]: creating pool ceph_test_rados_list_parallel.smithi100-80467 2024-02-14T17:47:57.566 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[81748]: created object 0... 2024-02-14T17:47:57.566 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[81748]: created object 25... 2024-02-14T17:47:57.566 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[81748]: created object 49... 2024-02-14T17:47:57.566 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[81748]: finishing. 2024-02-14T17:47:57.566 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_9_[81748]: shutting down. 2024-02-14T17:47:57.566 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.566 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[81749]: starting. 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[81749]: listing objects. 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[81749]: listed object 0... 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[81749]: listed object 25... 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[81749]: listed object 50... 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[81749]: listed object 75... 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[81749]: saw 99 objects 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_10_[81749]: shutting down. 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[81752]: starting. 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[81752]: removed 25 objects... 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[81752]: removed half of the objects 2024-02-14T17:47:57.567 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[81752]: removed 50 objects... 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[81752]: removed 50 objects 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_13_[81752]: shutting down. 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[81750]: starting. 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[81750]: added 25 objects... 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[81750]: added half of the objects 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[81750]: added 50 objects... 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[81750]: added 50 objects 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_11_[81750]: shutting down. 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[81751]: starting. 2024-02-14T17:47:57.568 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[81751]: added 25 objects... 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[81751]: added half of the objects 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[81751]: added 50 objects... 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[81751]: added 50 objects 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_12_[81751]: shutting down. 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[81860]: starting. 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[81860]: creating pool ceph_test_rados_list_parallel.smithi100-80467 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[81860]: created object 0... 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[81860]: created object 25... 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[81860]: created object 49... 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[81860]: finishing. 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_14_[81860]: shutting down. 2024-02-14T17:47:57.569 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.570 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.570 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.570 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:47:57.570 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[81861]: starting. 2024-02-14T17:47:57.570 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[81861]: listing objects. 2024-02-14T17:47:57.570 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[81861]: listed object 0... 2024-02-14T17:47:57.655 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[81861]: listed object 2 api_service: [==========] Running 4 tests from 1 test suite. 2024-02-14T17:47:57.655 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [----------] Global test environment set-up. 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5050 ms) 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ OK ] LibRadosService.RegisterLate (28 ms) 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: cluster: 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: id: d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: health: HEALTH_OK 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: services: 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-14T17:47:57.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: data: 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: usage: 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: pgs: 132 active+clean 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: io: 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: cluster: 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: id: d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: health: HEALTH_WARN 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 4 pool(s) do not have an application enabled 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: services: 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: data: 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: pools: 10 pools, 260 pgs 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: usage: 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:57.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: pgs: 49.231% pgs unknown 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 132 active+clean 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 128 unknown 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: io: 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: cluster: 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: id: d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: health: HEALTH_WARN 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 31 pool(s) do not have an application enabled 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: services: 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-14T17:47:57.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-14T17:47:57.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-14T17:47:57.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-14T17:47:57.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-14T17:47:57.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:47:57.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: data: 2024-02-14T17:47:57.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: pools: 37 pools, 1124 pgs 2024-02-14T17:47:57.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-14T17:47:57.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: usage: 231 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:47:57.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: pgs: 66.726% pgs unknown 2024-02-14T17:47:58.153 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: Running main() from gmock_main.cc 2024-02-14T17:47:58.154 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-02-14T17:47:58.154 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: [----------] Global test environment set-up. 2024-02-14T17:47:58.154 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-02-14T17:47:58.154 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-02-14T17:47:58.154 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (11785 ms) 2024-02-14T17:47:58.154 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: [----------] 1 test from neocls_handler_error (11785 ms total) 2024-02-14T17:47:58.154 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: 2024-02-14T17:47:58.154 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: [----------] Global test environment tear-down 2024-02-14T17:47:58.154 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: [==========] 1 test from 1 test suite ran. (11785 ms total) 2024-02-14T17:47:58.154 INFO:tasks.workunit.client.0.smithi100.stdout: handler_error: [ PASSED ] 1 test. 2024-02-14T17:47:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:47:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:47:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi100-80594-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi100-80594-1"}]': finished 2024-02-14T17:47:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi100-80625-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi100-80625-1"}]': finished 2024-02-14T17:47:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:58 smithi100 ceph-mon[30138]: osdmap e79: 8 total, 8 up, 8 in 2024-02-14T17:47:58.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:47:58.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:47:58.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi100-80594-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi100-80594-1"}]': finished 2024-02-14T17:47:58.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi100-80625-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi100-80625-1"}]': finished 2024-02-14T17:47:58.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:58 smithi100 ceph-mon[39557]: osdmap e79: 8 total, 8 up, 8 in 2024-02-14T17:47:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:47:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:47:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi100-80594-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi100-80594-1"}]': finished 2024-02-14T17:47:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi100-80625-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi100-80625-1"}]': finished 2024-02-14T17:47:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:58 smithi179 ceph-mon[33633]: osdmap e79: 8 total, 8 up, 8 in 2024-02-14T17:47:59.201 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-02-14T17:47:59.235 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-14T17:47:59.235 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-14T17:47:59.235 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-14T17:47:59.235 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-14T17:47:59.235 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (48 ms) 2024-02-14T17:47:59.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-14T17:47:59.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (61 ms) 2024-02-14T17:47:59.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-14T17:47:59.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (12924 ms) 2024-02-14T17:47:59.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (13033 ms total) 2024-02-14T17:47:59.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: 2024-02-14T17:47:59.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-14T17:47:59.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (13033 ms total) 2024-02-14T17:47:59.236 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-14T17:47:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-02-14T17:47:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-02-14T17:47:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-14T17:47:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-02-14T17:47:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-14T17:47:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-02-14T17:47:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-14T17:47:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (262 ms) 2024-02-14T17:47:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-14T17:47:59.237 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (9 ms) 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (25 ms) 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (19 ms) 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (16 ms) 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (13 ms) 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (10 ms) 2024-02-14T17:47:59.238 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (10 ms) 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (6 ms) 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (14 ms) 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-14T17:47:59.239 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (16 ms) 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (8 ms) 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (5 ms) 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (428 ms total) 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (12966 ms total) 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-02-14T17:47:59.240 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (12931 ms) 2024-02-14T17:47:59.241 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (12931 ms total) 2024-02-14T17:47:59.241 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: 2024-02-14T17:47:59.241 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-02-14T17:47:59.241 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (12931 ms total) 2024-02-14T17:47:59.241 INFO:tasks.workunit.client.0.smithi100.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-02-14T17:47:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: pgmap v77: 1092 pgs: 80 creating+peering, 994 active+clean, 18 unknown; 631 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 65 KiB/s wr, 121 op/s 2024-02-14T17:47:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3874719222' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:47:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:47:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-80161-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]': finished 2024-02-14T17:47:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: osdmap e80: 8 total, 8 up, 8 in 2024-02-14T17:47:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:47:59.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-79953-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:59.574 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:47:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:47:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: pgmap v77: 1092 pgs: 80 creating+peering, 994 active+clean, 18 unknown; 631 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 65 KiB/s wr, 121 op/s 2024-02-14T17:47:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3874719222' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-80161-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]': finished 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: osdmap e80: 8 total, 8 up, 8 in 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:47:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:47:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:47:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:47:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:47:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:47:59.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: pgmap v77: 1092 pgs: 80 creating+peering, 994 active+clean, 18 unknown; 631 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 65 KiB/s wr, 121 op/s 2024-02-14T17:47:59.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3874719222' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-14T17:47:59.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:47:59.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-80161-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]': finished 2024-02-14T17:47:59.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: osdmap e80: 8 total, 8 up, 8 in 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:47:59.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:47:59.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:47:59.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-79953-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:59.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:47:59.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-79953-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:47:59.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:47:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: Running main() from gmock_main.cc 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (58 ms) 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (55 ms) 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (12926 ms) 2024-02-14T17:47:59.783 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.137636+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi100-80161-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]': finished 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.149728+0000 mon.c [INF] from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.156301+0000 mon.c [INF] from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.171295+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.176334+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.433691+0000 mon.c [INF] from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.434284+0000 mon.c [INF] from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.434690+0000 mon.c [INF] from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.435095+0000 mon.c [INF] from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.435520+0000 mon.c [INF] from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.435923+0000 mon.c [INF] from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:47:59.784 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.436312+0000 mon.c [INF] from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:47:59.785 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.436680+0000 mon.c [INF] from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:47:59.785 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.437069+0000 mon.c [INF] from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:48:00.265 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.437475+0000 mon.c [INF] from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:48:00.265 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.462705+0000 mon.a [INF] from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.465636+0000 mon.a [INF] from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:58.465978+0000 mon.a [INF] from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-79953-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.180116+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.185785+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.185882+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.185960+0000 mon.a [INF] from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-79953-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.192745+0000 mon.b [INF] from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.194179+0000 mon.b [INF] from='client.? 172.21.15.100:0/2602052618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.195649+0000 mon.c [INF] from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.229634+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.229947+0000 mon.a [INF] from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-79953-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.287655+0000 mon.b [INF] from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:00.266 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.299377+0000 mon.c [INF] from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.267 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.776022+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: pgmap v80: 1116 pgs: 80 creating+peering, 834 active+clean, 202 unknown; 630 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 63 KiB/s wr, 117 op/s 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:48:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: 13.3 deep-scrub starts 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: 13.3 deep-scrub ok 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-79953-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: osdmap e81: 8 total, 8 up, 8 in 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2602052618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-79953-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:00.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.574 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.574 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.574 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:00.574 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:00.574 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:48:00.574 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:00.574 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:00.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:48:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: pgmap v80: 1116 pgs: 80 creating+peering, 834 active+clean, 202 unknown; 630 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 63 KiB/s wr, 117 op/s 2024-02-14T17:48:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:48:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: 13.3 deep-scrub starts 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: 13.3 deep-scrub ok 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-79953-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: osdmap e81: 8 total, 8 up, 8 in 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2602052618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-79953-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: pgmap v80: 1116 pgs: 80 creating+peering, 834 active+clean, 202 unknown; 630 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 63 KiB/s wr, 117 op/s 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:48:00.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: 13.3 deep-scrub starts 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: 13.3 deep-scrub ok 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi100-79953-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: osdmap e81: 8 total, 8 up, 8 in 2024-02-14T17:48:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:00.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2602052618' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:00.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:00.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:00.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-79953-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:00.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:00.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:00.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:00.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:01.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.776968+0000 mon.c [INF] from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:01.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.778692+0000 mon.a [INF] from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:01.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.781374+0000 mon.c [INF] from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:48:01.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.015525+0000 mon.a [INF] from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:01.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.016323+0000 mon.a [INF] from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:01.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:47:59.781970+0000 client.admin [INF] onexx 2024-02-14T17:48:01.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.269070+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]': finished 2024-02-14T17:48:01.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.269141+0000 mon.a [INF] from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:01.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.274367+0000 mon.c [INF] from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:01.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.536621+0000 mon.c [INF] from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.536886+0000 mon.c [INF] from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.537619+0000 mon.a [INF] from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-79752-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:01.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.539678+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:01.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.540361+0000 mon.c [INF] from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:01.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.781547+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:48:01.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.783343+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:01.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.783974+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:01.791 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.784245+0000 5... 2024-02-14T17:48:01.791 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[81861]: listed object 50... 2024-02-14T17:48:01.791 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[81861]: listed object 75... 2024-02-14T17:48:01.791 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[81861]: listed object 100... 2024-02-14T17:48:01.791 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[81861]: listed object 125... 2024-02-14T17:48:01.791 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[81861]: saw 150 objects 2024-02-14T17:48:01.791 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_15_[81861]: shutting down. 2024-02-14T17:48:01.791 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:48:01.791 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:48:01.791 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[81862]: starting. 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[81862]: added 25 objects... 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[81862]: added half of the objects 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[81862]: added 50 objects... 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[81862]: added 50 objects 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: process_16_[81862]: shutting down. 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******************************* 2024-02-14T17:48:01.792 INFO:tasks.workunit.client.0.smithi100.stdout: list_parallel: ******* SUCCESS ********** 2024-02-14T17:48:01.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:48:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: 13.8 deep-scrub starts 2024-02-14T17:48:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: 13.8 deep-scrub ok 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: onexx 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]': finished 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: osdmap e82: 8 total, 8 up, 8 in 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-79752-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:01.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:01.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: 13.8 deep-scrub starts 2024-02-14T17:48:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: 13.8 deep-scrub ok 2024-02-14T17:48:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: onexx 2024-02-14T17:48:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]': finished 2024-02-14T17:48:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: osdmap e82: 8 total, 8 up, 8 in 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-79752-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: 13.8 deep-scrub starts 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: 13.8 deep-scrub ok 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: onexx 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi100-80594-1"}]': finished 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3715527794' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: osdmap e82: 8 total, 8 up, 8 in 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-79752-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:01.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-14T17:48:01.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]: dispatch 2024-02-14T17:48:01.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:01.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:01.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:01.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.1 deep-scrub starts 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.1 deep-scrub ok 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.6 deep-scrub starts 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.6 deep-scrub ok 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.2 deep-scrub starts 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.2 deep-scrub ok 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.5 deep-scrub starts 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.5 deep-scrub ok 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.4 deep-scrub starts 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.4 deep-scrub ok 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.9 deep-scrub starts 2024-02-14T17:48:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: pgmap v82: 788 pgs: 80 creating+peering, 418 active+clean, 290 unknown; 591 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 19 KiB/s wr, 40 op/s 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: 13.9 deep-scrub ok 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: twoxx 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-79953-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-79953-7"}]': finished 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]': finished 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi100-80594-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2707637155' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3634364337' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:02.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: osdmap e83: 8 total, 8 up, 8 in 2024-02-14T17:48:02.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:02.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:48:02.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi100-80594-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:02.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.1 deep-scrub starts 2024-02-14T17:48:02.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.1 deep-scrub ok 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.6 deep-scrub starts 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.6 deep-scrub ok 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.2 deep-scrub starts 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.2 deep-scrub ok 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.5 deep-scrub starts 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.5 deep-scrub ok 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.4 deep-scrub starts 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.4 deep-scrub ok 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.9 deep-scrub starts 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: pgmap v82: 788 pgs: 80 creating+peering, 418 active+clean, 290 unknown; 591 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 19 KiB/s wr, 40 op/s 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: 13.9 deep-scrub ok 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:02.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: twoxx 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-79953-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-79953-7"}]': finished 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]': finished 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi100-80594-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2707637155' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3634364337' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: osdmap e83: 8 total, 8 up, 8 in 2024-02-14T17:48:02.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.1 deep-scrub starts 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.1 deep-scrub ok 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.6 deep-scrub starts 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.6 deep-scrub ok 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.2 deep-scrub starts 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.2 deep-scrub ok 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.5 deep-scrub starts 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.5 deep-scrub ok 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.4 deep-scrub starts 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.4 deep-scrub ok 2024-02-14T17:48:02.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.9 deep-scrub starts 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: pgmap v82: 788 pgs: 80 creating+peering, 418 active+clean, 290 unknown; 591 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 2.9 KiB/s rd, 19 KiB/s wr, 40 op/s 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: 13.9 deep-scrub ok 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: twoxx 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi100-79953-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi100-79953-7"}]': finished 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi100-80594-1"}]': finished 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi100-80594-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:02.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:02.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2707637155' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3634364337' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:02.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: osdmap e83: 8 total, 8 up, 8 in 2024-02-14T17:48:02.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.905 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.905 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.905 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.905 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi100-80594-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi100-80594-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:02.906 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: pgmap v85: 924 pgs: 229 creating+peering, 559 active+clean, 136 unknown; 145 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 36 MiB/s wr, 417 op/s 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-79752-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-79752-10"}]': finished 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi100-80594-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: osdmap e84: 8 total, 8 up, 8 in 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:03.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:03.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:03.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:03.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:03.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: pgmap v85: 924 pgs: 229 creating+peering, 559 active+clean, 136 unknown; 145 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 36 MiB/s wr, 417 op/s 2024-02-14T17:48:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-79752-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-79752-10"}]': finished 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi100-80594-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:03.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: osdmap e84: 8 total, 8 up, 8 in 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:03.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: pgmap v85: 924 pgs: 229 creating+peering, 559 active+clean, 136 unknown; 145 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 36 MiB/s wr, 417 op/s 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-79752-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-79752-10"}]': finished 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi100-80594-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:48:03.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: osdmap e84: 8 total, 8 up, 8 in 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:03.904 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:03.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:03.905 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[80487]: starting. 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[80487]: creating pool ceph_test_rados_open_pools_parallel.smithi100-80440 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[80487]: created object 0... 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[80487]: created object 25... 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[80487]: created object 49... 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[80487]: finishing. 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_1_[80487]: shutting down. 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_2_[80493]: starting. 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_2_[80493]: rados_pool_create. 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_2_[80493]: rados_ioctx_create. 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_2_[80493]: shutting down. 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: ******************************* 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[82010]: starting. 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[82010]: creating pool ceph_test_rados_open_pools_parallel.smithi100-80440 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[82010]: created object 0... 2024-02-14T17:48:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[82010]: created object 25... 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[82010]: created object 49... 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[82010]: finishing. 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_3_[82010]: shutting down. 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: ******************************* 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_4_[82011]: starting. 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_4_[82011]: rados_pool_create. 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_4_[82011]: rados_ioctx_create. 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: process_4_[82011]: shutting down. 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: ******************************* 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: ******************************* 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[80498]: starting. 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[80498]: creating pool ceph_test_rados_delete_pools_parallel.smithi100-80393 2024-02-14T17:48:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[80498]: created object 0... 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[80498]: created object 25... 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[80498]: created object 49... 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[80498]: finishing. 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_1_[80498]: shutting down. 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_2_[80502]: starting. 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_2_[80502]: deleting pool ceph_test_rados_delete_pools_parallel.smithi100-80393 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_2_[80502]: shutting down. 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******************************* 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[81997]: starting. 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[81997]: creating pool ceph_test_rados_delete_pools_parallel.smithi100-80393 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[81997]: created object 0... 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[81997]: created object 25... 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[81997]: created object 49... 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[81997]: finishing. 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_3_[81997]: shutting down. 2024-02-14T17:48:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******************************* 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[81999]: starting. 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[81999]: listing objects. 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[81999]: listed object 0... 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[81999]: listed object 25... 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[81999]: saw 50 objects 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_5_[81999]: shutting down. 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******************************* 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_4_[81998]: starting. 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_4_[81998]: deleting pool ceph_test_rados_delete_pools_parallel.smithi100-80393 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: process_4_[81998]: shutting down. 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******************************* 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******************************* 2024-02-14T17:48:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-14T17:48:04.523 INFO:tasks.workunit.client.0.smithi100.stdout:mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi100-79590-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:04.523 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:00.784849+0000 mon.b [INF] from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:04.523 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:01.258187+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:04.523 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.275336+0000 mon.c [INF] from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:04.523 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.518351+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-14T17:48:04.524 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.522850+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:04.524 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.523648+0000 mon.a [INF] from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi100-79752-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi100-79752-10"}]': finished 2024-02-14T17:48:04.524 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.523730+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi100-79612-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:04.524 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.523811+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:04.524 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.523886+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:04.524 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.523959+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi100-79629-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:04.524 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.524023+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi100-80594-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:04.524 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.530670+0000 mon.c [INF] from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:04.524 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.531952+0000 mon.c [INF] from='client.? 172.21.15.100:0/2658288504' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:48:04.524 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.532427+0000 mon.b [INF] from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:04.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.532644+0000 mon.b [INF] from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:04.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.536244+0000 mon.c [INF] from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:04.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.537088+0000 mon.c [INF] from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:04.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.813895+0000 mon.c [INF] from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:04.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.817467+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:04.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.822479+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-14T17:48:04.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.822953+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:04.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:02.823315+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:04.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.075068+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:04.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.320694+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.320887+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.321090+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.321332+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.321546+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.552156+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.552224+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]': finished 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.552305+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]': finished 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.552385+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.552465+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.552530+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.552609+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.558358+0000 mon.c [INF] from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.558634+0000 mon.c [INF] from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.558944+0000 mon.c [INF] from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.558959+0000 mon.b [INF] from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:04.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:03.830390+0000 mon.b [INF] from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:04.584 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:04.324809+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:04.584 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:04.325220+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:04.584 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:04.325814+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-14T17:48:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]': finished 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]': finished 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: osdmap e85: 8 total, 8 up, 8 in 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:04.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:04.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:04.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:04.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-14T17:48:04.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]': finished 2024-02-14T17:48:04.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]': finished 2024-02-14T17:48:04.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:04.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:04.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:04.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:04.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: osdmap e85: 8 total, 8 up, 8 in 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:04.903 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:05.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi100-79612-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]': finished 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi100-80625-1"}]': finished 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi100-79691-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1569892570' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: osdmap e85: 8 total, 8 up, 8 in 2024-02-14T17:48:05.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:05.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]: dispatch 2024-02-14T17:48:05.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:05.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:05.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:06.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:48:06.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:05 smithi179 ceph-mon[33633]: threexx 2024-02-14T17:48:06.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:48:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:48:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:05 smithi100 ceph-mon[30138]: threexx 2024-02-14T17:48:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:48:06.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:48:06.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:05 smithi100 ceph-mon[39557]: threexx 2024-02-14T17:48:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:48:06.834 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:04.326423+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Lib cmd: Running main() from gmock_main.cc 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [----------] Global test environment set-up. 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (9497 ms) 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (3386 ms) 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (7627 ms) 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [----------] 3 tests from NeoRadosCmd (20510 ms total) 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [----------] Global test environment tear-down 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [==========] 3 tests from 1 test suite ran. (20510 ms total) 2024-02-14T17:48:06.835 INFO:tasks.workunit.client.0.smithi100.stdout: cmd: [ PASSED ] 3 tests. 2024-02-14T17:48:06.836 INFO:tasks.workunit.client.0.smithi100.stdout: cls: Running main() from gmock_main.cc 2024-02-14T17:48:06.836 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-02-14T17:48:06.836 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [----------] Global test environment set-up. 2024-02-14T17:48:06.836 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-02-14T17:48:06.836 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-02-14T17:48:06.836 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [ OK ] NeoRadosCls.DNE (12881 ms) 2024-02-14T17:48:06.836 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-02-14T17:48:06.836 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (7638 ms) 2024-02-14T17:48:06.836 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [----------] 2 tests from NeoRadosCls (20519 ms total) 2024-02-14T17:48:06.837 INFO:tasks.workunit.client.0.smithi100.stdout: cls: 2024-02-14T17:48:06.837 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [----------] Global test environment tear-down 2024-02-14T17:48:06.837 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [==========] 2 tests from 1 test suite ran. (20519 ms total) 2024-02-14T17:48:06.837 INFO:tasks.workunit.client.0.smithi100.stdout: cls: [ PASSED ] 2 tests. 2024-02-14T17:48:06.844 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: Running main() from gmock_main.cc 2024-02-14T17:48:06.844 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-02-14T17:48:06.844 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] Global test environment set-up. 2024-02-14T17:48:06.844 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-02-14T17:48:06.844 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStat.Stat (301 ms) 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (5 ms) 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStat.StatNS (19 ms) 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (4 ms) 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] 5 tests from LibRadosStat (330 ms total) 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (382 ms) 2024-02-14T17:48:06.845 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-14T17:48:06.846 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (21 ms) 2024-02-14T17:48:06.846 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-14T17:48:06.846 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (159 ms) 2024-02-14T17:48:06.846 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-14T17:48:06.846 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (248 ms) 2024-02-14T17:48:06.846 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (810 ms total) 2024-02-14T17:48:06.846 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: 2024-02-14T17:48:06.846 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [----------] Global test environment tear-down 2024-02-14T17:48:06.846 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (20725 ms total) 2024-02-14T17:48:06.846 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat: [ PASSED ] 9 tests. 2024-02-14T17:48:06.851 INFO:tasks.workunit.client.0.smithi100.stdout: list: Running main() from gmock_main.cc 2024-02-14T17:48:06.851 INFO:tasks.workunit.client.0.smithi100.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-02-14T17:48:06.851 INFO:tasks.workunit.client.0.smithi100.stdout: list: [----------] Global test environment set-up. 2024-02-14T17:48:06.851 INFO:tasks.workunit.client.0.smithi100.stdout: list: [----------] 3 tests from NeoradosList 2024-02-14T17:48:06.851 INFO:tasks.workunit.client.0.smithi100.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-02-14T17:48:06.851 INFO:tasks.workunit.client.0.smithi100.stdout: list: [ OK ] NeoradosList.ListObjects (10949 ms) 2024-02-14T17:48:06.852 INFO:tasks.workunit.client.0.smithi100.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-02-14T17:48:06.852 INFO:tasks.workunit.client.0.smithi100.stdout: list: [ OK ] NeoradosList.ListObjectsNS (5534 ms) 2024-02-14T17:48:06.852 INFO:tasks.workunit.client.0.smithi100.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-02-14T17:48:06.852 INFO:tasks.workunit.client.0.smithi100.stdout: list: [ OK ] NeoradosList.ListObjectsMany (4021 ms) 2024-02-14T17:48:06.852 INFO:tasks.workunit.client.0.smithi100.stdout: list: [----------] 3 tests from NeoradosList (20504 ms total) 2024-02-14T17:48:06.852 INFO:tasks.workunit.client.0.smithi100.stdout: list: 2024-02-14T17:48:06.852 INFO:tasks.workunit.client.0.smithi100.stdout: list: [----------] Global test environment tear-down 2024-02-14T17:48:06.852 INFO:tasks.workunit.client.0.smithi100.stdout: list: [==========] 3 tests from 1 test suite ran. (20504 ms total) 2024-02-14T17:48:06.852 INFO:tasks.workunit.client.0.smithi100.stdout: list: [ PASSED ] 3 tests. 2024-02-14T17:48:07.025 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: Running main() from gmock_main.cc 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: seed 79645 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (118 ms) 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (7 ms) 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (3 ms) 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-14T17:48:07.026 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (2 ms) 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2 ms) 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (2 ms) 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (2 ms) 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (3 ms) 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (2 ms) 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (2 ms) 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-14T17:48:07.027 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (4 ms) 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (3 ms) 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (2 ms) 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (2 ms) 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (7 ms) 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (3 ms) 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-14T17:48:07.028 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (4 ms) 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (4 ms) 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (5 ms) 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (177 ms total) 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (2364 ms) 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (53 ms) 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (4 ms) 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-14T17:48:07.029 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (4 ms) 2024-02-14T17:48:07.030 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-14T17:48:07.030 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (5 ms) 2024-02-14T17:48:07.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: pgmap v88: 844 pgs: 101 creating+peering, 479 active+clean, 264 unknown; 145 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 503 op/s 2024-02-14T17:48:07.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: 13.0 deep-scrub starts 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: 13.0 deep-scrub ok 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]': finished 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]': finished 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]': finished 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]': finished 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]': finished 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]': finished 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: 13.7 deep-scrub starts 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: 13.7 deep-scrub ok 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: osdmap e86: 8 total, 8 up, 8 in 2024-02-14T17:48:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:48:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]': finished 2024-02-14T17:48:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]': finished 2024-02-14T17:48:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-14T17:48:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:48:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: fourxx 2024-02-14T17:48:07.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: osdmap e87: 8 total, 8 up, 8 in 2024-02-14T17:48:07.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: pgmap v88: 844 pgs: 101 creating+peering, 479 active+clean, 264 unknown; 145 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 503 op/s 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: 13.0 deep-scrub starts 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: 13.0 deep-scrub ok 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]': finished 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]': finished 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]': finished 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]': finished 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]': finished 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]': finished 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: 13.7 deep-scrub starts 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: 13.7 deep-scrub ok 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:07.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: osdmap e86: 8 total, 8 up, 8 in 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]': finished 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]': finished 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: fourxx 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: osdmap e87: 8 total, 8 up, 8 in 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1516987140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-79590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi100-80625-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:07.154 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:07.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:48:07.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-79590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:07.155 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:07.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1516987140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-79590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:07.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi100-80625-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:07.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:07.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:48:07.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-79590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:07.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:07.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: pgmap v88: 844 pgs: 101 creating+peering, 479 active+clean, 264 unknown; 145 MiB data, 713 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 503 op/s 2024-02-14T17:48:07.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: 13.0 deep-scrub starts 2024-02-14T17:48:07.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: 13.0 deep-scrub ok 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi100-79629-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi100-79629-16"}]': finished 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi100-79645-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi100-79645-22"}]': finished 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi100-80594-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi100-80594-2"}]': finished 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi100-80625-1"}]': finished 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-4", "overlaypool": "test-rados-api-smithi100-79838-4-cache"}]': finished 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]': finished 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: 13.7 deep-scrub starts 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: 13.7 deep-scrub ok 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: osdmap e86: 8 total, 8 up, 8 in 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:07.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-4-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi100-80125-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi100-80125-7"}]': finished 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]': finished 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: fourxx 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: osdmap e87: 8 total, 8 up, 8 in 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1516987140' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-79590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi100-80625-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-14T17:48:07.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-79590-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:07.325 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:08.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: Health check update: 20 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:08.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-79953-7"}]': finished 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-4-cache", "mode": "writeback"}]': finished 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi100-80625-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]': finished 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-79590-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: osdmap e88: 8 total, 8 up, 8 in 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3992521320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi100-80625-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi100-80625-2"}]: dispatch 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:08.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-4"}]: dispatch 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: Health check update: 20 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-79953-7"}]': finished 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-4-cache", "mode": "writeback"}]': finished 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi100-80625-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]': finished 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-79590-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:08.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: osdmap e88: 8 total, 8 up, 8 in 2024-02-14T17:48:08.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3992521320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:08.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi100-80625-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi100-80625-2"}]: dispatch 2024-02-14T17:48:08.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:08.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:08.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:08.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:08.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-4"}]: dispatch 2024-02-14T17:48:08.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:08.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: Health check update: 20 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi100-79953-7"}]': finished 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-4-cache", "mode": "writeback"}]': finished 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi100-80625-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi100-80161-1"}]': finished 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi100-79590-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1038626748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: osdmap e88: 8 total, 8 up, 8 in 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3992521320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:08.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi100-80625-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi100-80625-2"}]: dispatch 2024-02-14T17:48:08.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:08.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]: dispatch 2024-02-14T17:48:08.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:08.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:08.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-4"}]: dispatch 2024-02-14T17:48:08.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:08.832 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LRadosLockEC_smithi100-79691-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:08.832 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:04.327122+0000 mon.a [INF] from='client.? 172.21.15.100:0/3499429078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi100-79953-7"}]: dispatch 2024-02-14T17:48:08.832 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:04.522704+0000 mon.c [INF] from='client.? 172.21.15.100:0/336044595' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:48:08.832 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:04.522854+0000 client.admin [INF] threexx 2024-02-14T17:48:08.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: got: 2024-02-14T17:48:04.586603+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-14T17:48:08.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (9630 ms) 2024-02-14T17:48:08.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (22669 ms total) 2024-02-14T17:48:08.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: 2024-02-14T17:48:08.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-14T17:48:08.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (22669 ms total) 2024-02-14T17:48:08.833 INFO:tasks.workunit.client.0.smithi100.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-14T17:48:08.898 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: seed 79752 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (110 ms) 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (15 ms) 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1123 ms) 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1025 ms) 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (3 ms) 2024-02-14T17:48:08.899 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (4 ms) 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (3 ms) 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (3 ms) 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2286 ms total) 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (839 ms) 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (6 ms) 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-14T17:48:08.900 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1005 ms) 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1005 ms) 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (3 ms) 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (5 ms) 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (4 ms) 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (2870 ms total) 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (22848 ms total) 2024-02-14T17:48:08.901 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-14T17:48:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: pgmap v91: 980 pgs: 168 creating+peering, 812 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 10 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-02-14T17:48:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: pool 'PoolQuotaPP_smithi100-79612-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-14T17:48:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:48:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10"}]': finished 2024-02-14T17:48:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]': finished 2024-02-14T17:48:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: osdmap e89: 8 total, 8 up, 8 in 2024-02-14T17:48:10.205 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1846315564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:10.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-4"}]: dispatch 2024-02-14T17:48:10.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:10.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:10.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:10.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi100-80625-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi100-80625-2"}]': finished 2024-02-14T17:48:10.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-79752-10"}]': finished 2024-02-14T17:48:10.206 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-4"}]': finished 2024-02-14T17:48:10.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: pgmap v91: 980 pgs: 168 creating+peering, 812 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 10 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-02-14T17:48:10.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:10.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: pool 'PoolQuotaPP_smithi100-79612-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-14T17:48:10.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:48:10.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10"}]': finished 2024-02-14T17:48:10.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]': finished 2024-02-14T17:48:10.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:10.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: osdmap e89: 8 total, 8 up, 8 in 2024-02-14T17:48:10.207 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1846315564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-4"}]: dispatch 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi100-80625-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi100-80625-2"}]': finished 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-79752-10"}]': finished 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-4"}]': finished 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:10.208 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: osdmap e90: 8 total, 8 up, 8 in 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-80034-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-80034-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:10.209 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: osdmap e90: 8 total, 8 up, 8 in 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-80034-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-80034-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.210 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:10.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:10.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:10.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:10.211 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: pgmap v91: 980 pgs: 168 creating+peering, 812 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 10 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: pool 'PoolQuotaPP_smithi100-79612-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi100-79752-10"}]': finished 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi100-80161-1"}]': finished 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: osdmap e89: 8 total, 8 up, 8 in 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1846315564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-79752-10"}]: dispatch 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-4"}]: dispatch 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi100-80625-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi100-80625-2"}]': finished 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3603027742' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi100-79752-10"}]': finished 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-4"}]': finished 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: osdmap e90: 8 total, 8 up, 8 in 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-80034-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2302883194' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:10.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:10.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:10.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-80034-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:10.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:10.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]: dispatch 2024-02-14T17:48:10.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:10.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:10.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:10.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:10.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:48:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:10.946 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: Running main() from gmock_main.cc 2024-02-14T17:48:10.946 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-02-14T17:48:10.946 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [----------] Global test environment set-up. 2024-02-14T17:48:10.946 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-02-14T17:48:10.946 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-02-14T17:48:10.946 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (15146 ms) 2024-02-14T17:48:10.946 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-02-14T17:48:10.947 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (9431 ms) 2024-02-14T17:48:10.947 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (24577 ms total) 2024-02-14T17:48:10.947 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: 2024-02-14T17:48:10.947 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [----------] Global test environment tear-down 2024-02-14T17:48:10.947 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (24577 ms total) 2024-02-14T17:48:10.947 INFO:tasks.workunit.client.0.smithi100.stdout: ec_io: [ PASSED ] 2 tests. 2024-02-14T17:48:10.950 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: seed 80125 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (1174 ms) 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (116 ms) 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-14T17:48:10.951 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (6 ms) 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (26 ms) 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (1323 ms total) 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1171 ms) 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (3 ms) 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (4 ms) 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (11 ms) 2024-02-14T17:48:10.952 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1189 ms total) 2024-02-14T17:48:10.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: 2024-02-14T17:48:10.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-14T17:48:10.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (24775 ms total) 2024-02-14T17:48:10.953 INFO:tasks.workunit.client.0.smithi100.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-14T17:48:10.961 INFO:tasks.workunit.client.0.smithi100.stdout:ibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-14T17:48:10.961 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (5 ms) 2024-02-14T17:48:10.961 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-14T17:48:10.961 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-02-14T17:48:10.961 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-14T17:48:10.961 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-02-14T17:48:10.961 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-14T17:48:10.961 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (5 ms) 2024-02-14T17:48:10.961 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (4 ms) 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (8 ms) 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (3 ms) 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (6 ms) 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (6 ms) 2024-02-14T17:48:10.962 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (2488 ms total) 2024-02-14T17:48:10.963 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: 2024-02-14T17:48:10.963 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-14T17:48:10.963 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (24952 ms total) 2024-02-14T17:48:10.963 INFO:tasks.workunit.client.0.smithi100.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-14T17:48:10.968 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: Running main() from gmock_main.cc 2024-02-14T17:48:10.968 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] Global test environment set-up. 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (296 ms) 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: no timeout :/ 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: no timeout :/ 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: no timeout :/ 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: no timeout :/ 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: no timeout :/ 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (45 ms) 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-14T17:48:10.969 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (13 ms) 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.Checksum (4 ms) 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (5 ms) 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (5 ms) 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (5 ms) 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (3 ms) 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.TruncTest (5 ms) 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (4 ms) 2024-02-14T17:48:10.970 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (6 ms) 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.RmXattr (14 ms) 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIo.XattrIter (6 ms) 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] 14 tests from LibRadosIo (411 ms total) 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (2393 ms) 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (10 ms) 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (9 ms) 2024-02-14T17:48:10.971 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (8 ms) 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (4 ms) 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (6 ms) 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] 10 tests from LibRadosIoEC (2451 ms total) 2024-02-14T17:48:10.972 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: 2024-02-14T17:48:10.973 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [----------] Global test environment tear-down 2024-02-14T17:48:10.973 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [==========] 24 tests from 2 test suites ran. (24967 ms total) 2024-02-14T17:48:10.973 INFO:tasks.workunit.client.0.smithi100.stdout: api_io: [ PASSED ] 24 tests. 2024-02-14T17:48:11.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: pgmap v94: 844 pgs: 160 unknown, 136 creating+peering, 548 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:48:11.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:11.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:48:11.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:11.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]': finished 2024-02-14T17:48:11.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]': finished 2024-02-14T17:48:11.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:11.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi100-80594-2"}]': finished 2024-02-14T17:48:11.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]': finished 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]': finished 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-80161-6", "pg_num": 4}]: dispatch 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: osdmap e91: 8 total, 8 up, 8 in 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3554053269' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-79590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:11.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:11.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: pgmap v94: 844 pgs: 160 unknown, 136 creating+peering, 548 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:48:11.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:11.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:48:11.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:11.214 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]': finished 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]': finished 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi100-80594-2"}]': finished 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]': finished 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]': finished 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-80161-6", "pg_num": 4}]: dispatch 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: osdmap e91: 8 total, 8 up, 8 in 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3554053269' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-79590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:11.215 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:11.216 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:11.216 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: pgmap v94: 844 pgs: 160 unknown, 136 creating+peering, 548 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:48:11.216 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:11.216 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:48:11.216 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:11.216 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi100-80125-7"}]': finished 2024-02-14T17:48:11.216 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-4", "tierpool": "test-rados-api-smithi100-79838-4-cache"}]': finished 2024-02-14T17:48:11.216 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:11.216 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi100-80594-2"}]': finished 2024-02-14T17:48:11.216 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi100-79629-16"}]': finished 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi100-79645-22"}]': finished 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1972283619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1763100167' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3080383670' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/374263194' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-80161-6", "pg_num": 4}]: dispatch 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: osdmap e91: 8 total, 8 up, 8 in 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3554053269' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-79590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]: dispatch 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi100-80594-2"}]: dispatch 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]: dispatch 2024-02-14T17:48:11.217 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]: dispatch 2024-02-14T17:48:11.939 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: pgmap v97: 820 pgs: 296 unknown, 8 creating+peering, 516 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-80161-6", "pg_num": 4}]: dispatch 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-79590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-80034-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]': finished 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]': finished 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi100-80594-2"}]': finished 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]': finished 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]': finished 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-80161-6", "pg_num": 4}]': finished 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: osdmap e92: 8 total, 8 up, 8 in 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1096489766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-79925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/774587915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi100-80625-2"}]: dispatch 2024-02-14T17:48:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-79925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:12.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: pgmap v97: 820 pgs: 296 unknown, 8 creating+peering, 516 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-80161-6", "pg_num": 4}]: dispatch 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-79590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-80034-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]': finished 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]': finished 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi100-80594-2"}]': finished 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]': finished 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]': finished 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-80161-6", "pg_num": 4}]': finished 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: osdmap e92: 8 total, 8 up, 8 in 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1096489766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-79925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/774587915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:12.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi100-80625-2"}]: dispatch 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-79925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: pgmap v97: 820 pgs: 296 unknown, 8 creating+peering, 516 active+clean; 145 MiB data, 747 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-80161-6", "pg_num": 4}]: dispatch 2024-02-14T17:48:12.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-79590-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi100-80034-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]': finished 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi100-79629-16"}]': finished 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi100-80594-2"}]': finished 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi100-79645-22"}]': finished 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi100-80125-7"}]': finished 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-80161-6", "pg_num": 4}]': finished 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: osdmap e92: 8 total, 8 up, 8 in 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1096489766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-79925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/774587915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi100-80625-2"}]: dispatch 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-79925-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:12.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:12.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:12.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:12.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:12.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:13.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: Running main() from gmock_main.cc 2024-02-14T17:48:13.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-14T17:48:13.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] Global test environment set-up. 2024-02-14T17:48:13.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-14T17:48:13.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-14T17:48:13.076 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (112 ms) 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.LockShared (24 ms) 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1273 ms) 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1004 ms) 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4 ms) 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.Unlock (3 ms) 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (3 ms) 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-14T17:48:13.077 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] 8 tests from LibRadosLock (2426 ms total) 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1123 ms) 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (35 ms) 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1039 ms) 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1082 ms) 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (3 ms) 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-14T17:48:13.078 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (4 ms) 2024-02-14T17:48:13.079 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-14T17:48:13.079 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (4 ms) 2024-02-14T17:48:13.079 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-14T17:48:13.079 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (2 ms) 2024-02-14T17:48:13.079 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3292 ms total) 2024-02-14T17:48:13.079 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: 2024-02-14T17:48:13.079 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [----------] Global test environment tear-down 2024-02-14T17:48:13.079 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (27045 ms total) 2024-02-14T17:48:13.079 INFO:tasks.workunit.client.0.smithi100.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-14T17:48:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-79590-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi100-80625-2"}]': finished 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]': finished 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-79925-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: osdmap e93: 8 total, 8 up, 8 in 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi100-80625-2"}]: dispatch 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "overlaypool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1422317054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "overlaypool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:13.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-79590-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi100-80625-2"}]': finished 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]': finished 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-79925-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: osdmap e93: 8 total, 8 up, 8 in 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi100-80625-2"}]: dispatch 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "overlaypool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1422317054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "overlaypool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:13.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi100-79590-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi100-80625-2"}]': finished 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi100-79691-10"}]': finished 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi100-79925-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: osdmap e93: 8 total, 8 up, 8 in 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi100-80625-2"}]: dispatch 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160808967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]: dispatch 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "overlaypool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1422317054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "overlaypool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: pool: Running main() from gmock_main.cc 2024-02-14T17:48:14.065 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [----------] Global test environment set-up. 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ OK ] NeoRadosPools.PoolList (10010 ms) 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (5357 ms) 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (2045 ms) 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (5035 ms) 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (3097 ms) 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2094 ms) 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [----------] 6 tests from NeoRadosPools (27639 ms total) 2024-02-14T17:48:14.066 INFO:tasks.workunit.client.0.smithi100.stdout: pool: 2024-02-14T17:48:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [----------] Global test environment tear-down 2024-02-14T17:48:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [==========] 6 tests from 1 test suite ran. (27639 ms total) 2024-02-14T17:48:14.067 INFO:tasks.workunit.client.0.smithi100.stdout: pool: [ PASSED ] 6 tests. 2024-02-14T17:48:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: pgmap v100: 688 pgs: 76 creating+peering, 96 unknown, 516 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi100-80625-2"}]': finished 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]': finished 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "overlaypool": "test-rados-api-smithi100-80161-6"}]': finished 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: osdmap e94: 8 total, 8 up, 8 in 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-80161-6", "mode": "writeback"}]: dispatch 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-80161-6", "mode": "writeback"}]: dispatch 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi100-80625-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:14.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:14.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:14 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: pgmap v100: 688 pgs: 76 creating+peering, 96 unknown, 516 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-02-14T17:48:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi100-80625-2"}]': finished 2024-02-14T17:48:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]': finished 2024-02-14T17:48:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "overlaypool": "test-rados-api-smithi100-80161-6"}]': finished 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: osdmap e94: 8 total, 8 up, 8 in 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-80161-6", "mode": "writeback"}]: dispatch 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-80161-6", "mode": "writeback"}]: dispatch 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi100-80625-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: pgmap v100: 688 pgs: 76 creating+peering, 96 unknown, 516 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/466004640' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi100-80625-2"}]': finished 2024-02-14T17:48:14.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi100-79691-10"}]': finished 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "overlaypool": "test-rados-api-smithi100-80161-6"}]': finished 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: osdmap e94: 8 total, 8 up, 8 in 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-80161-6", "mode": "writeback"}]: dispatch 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-80161-6", "mode": "writeback"}]: dispatch 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi100-80625-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:14.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:14 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-02-14T17:48:15.148 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify_test_cb 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (313 ms) 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94434344933232 err -107 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (11 ms) 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: waiting up to watch_notify2_test_errcb300 cookie for disconnect notification ...94434345636912 err 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: -107 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1012 ms) 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-14T17:48:15.149 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 24692 notify_id 339302416385 cookie 94434345636912 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (4 ms) 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 24692 notify_id 339302416384 cookie 94434345636912 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (4 ms) 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 24692 notify_id 339302416385 cookie 94434345691712 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (5 ms) 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 24692 notify_id 339302416385 cookie 94434345676368 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 24692 notify_id 339302416385 cookie 94434345691712 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (6 ms) 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 24692 notify_id 339302416385 cookie 94434345691712 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 24692 notify_id 343597383682 cookie 94434345691712 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3004 ms) 2024-02-14T17:48:15.150 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94434345691712 err -107 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_cb from 24692 notify_id 369367187460 cookie 94434345691712 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5008 ms) 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94434345691712 err -107 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1004 ms) 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (10371 ms total) 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: watch_notify_test_cb 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1084 ms) 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1084 ms total) 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: 2024-02-14T17:48:15.151 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-14T17:48:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (29007 ms total) 2024-02-14T17:48:15.152 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-02-14T17:48:15.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:15.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T17:48:15.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]': finished 2024-02-14T17:48:15.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-80161-6", "mode": "writeback"}]': finished 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi100-80625-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: osdmap e95: 8 total, 8 up, 8 in 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4111953941' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi100-80625-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi100-80625-3"}]: dispatch 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]': finished 2024-02-14T17:48:15.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-80161-6", "mode": "writeback"}]': finished 2024-02-14T17:48:15.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi100-80625-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:15.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: osdmap e95: 8 total, 8 up, 8 in 2024-02-14T17:48:15.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:15.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4111953941' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi100-80625-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi100-80625-3"}]: dispatch 2024-02-14T17:48:15.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:15.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T17:48:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi100-80034-12"}]': finished 2024-02-14T17:48:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-80161-6", "mode": "writeback"}]': finished 2024-02-14T17:48:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi100-80625-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: osdmap e95: 8 total, 8 up, 8 in 2024-02-14T17:48:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1450150886' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4111953941' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi100-80625-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi100-80625-3"}]: dispatch 2024-02-14T17:48:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]: dispatch 2024-02-14T17:48:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: pgmap v103: 584 pgs: 36 creating+peering, 128 unknown, 420 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-14T17:48:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]': finished 2024-02-14T17:48:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2099639361' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: osdmap e96: 8 total, 8 up, 8 in 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: pgmap v103: 584 pgs: 36 creating+peering, 128 unknown, 420 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]': finished 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2099639361' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: osdmap e96: 8 total, 8 up, 8 in 2024-02-14T17:48:16.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:16.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:16.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:16.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: pgmap v103: 584 pgs: 36 creating+peering, 128 unknown, 420 active+clean; 145 MiB data, 796 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-14T17:48:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi100-80034-12"}]': finished 2024-02-14T17:48:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2099639361' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: osdmap e96: 8 total, 8 up, 8 in 2024-02-14T17:48:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (3495 ms) 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (5 ms) 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (3500 ms total) 2024-02-14T17:48:17.242 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (63 ms) 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4211 ms) 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (5 ms) 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (4 ms) 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94818455954752 notify_id 408021893120 notifier_gid 25289 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (6 ms) 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94818455967776 notify_id 408021893123 notifier_gid 25289 2024-02-14T17:48:17.243 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (5 ms) 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94818455967776 notify_id 408021893124 notifier_gid 25289 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (5 ms) 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94818455967776 notify_id 408021893121 notifier_gid 25289 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (4 ms) 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94818455967776 notify_id 408021893120 notifier_gid 25289 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (6 ms) 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94818455967776 notify_id 408021893122 notifier_gid 25289 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (5 ms) 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-14T17:48:17.244 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: trying... 2024-02-14T17:48:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94818455967776 notify_id 408021893123 notifier_gid 25289 2024-02-14T17:48:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: timed out 2024-02-14T17:48:17.245 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: flushing 2024-02-14T17:48:17.494 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-14T17:48:17.494 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-14T17:48:17.494 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-14T17:48:17.494 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5042 ms) 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (1093 ms) 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20040 ms) 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: attempt 0 of 20 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5103 ms) 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (31278 ms total) 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (31278 ms total) 2024-02-14T17:48:17.495 INFO:tasks.workunit.client.0.smithi100.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-14T17:48:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: pgmap v105: 616 pgs: 128 creating+peering, 488 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-14T17:48:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi100-80625-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi100-80625-3"}]': finished 2024-02-14T17:48:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: osdmap e97: 8 total, 8 up, 8 in 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2250462780' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-79590-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/438685784' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-79590-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: pgmap v107: 720 pgs: 136 unknown, 128 creating+peering, 456 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-14T17:48:17.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[39557]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: pgmap v105: 616 pgs: 128 creating+peering, 488 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi100-80625-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi100-80625-3"}]': finished 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: osdmap e97: 8 total, 8 up, 8 in 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2250462780' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-79590-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/438685784' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-79590-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: pgmap v107: 720 pgs: 136 unknown, 128 creating+peering, 456 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:17.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:17.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-14T17:48:17.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:17 smithi100 ceph-mon[30138]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:17.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 19.573% pgs not active 2024-02-14T17:48:17.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 750 unknown 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 220 creating+peering 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 154 active+clean 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: io: 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ OK ] LibRadosService.StatusFormat (6390 ms) 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ OK ] LibRadosService.Status (20020 ms) 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [----------] 4 tests from LibRadosService (31488 ms total) 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [----------] Global test environment tear-down 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [==========] 4 tests from 1 test suite ran. (31489 ms total) 2024-02-14T17:48:17.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_service: [ PASSED ] 4 tests. 2024-02-14T17:48:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: pgmap v105: 616 pgs: 128 creating+peering, 488 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-14T17:48:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi100-80625-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi100-80625-3"}]': finished 2024-02-14T17:48:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: osdmap e97: 8 total, 8 up, 8 in 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2250462780' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-79590-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/438685784' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-79590-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: pgmap v107: 720 pgs: 136 unknown, 128 creating+peering, 456 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-14T17:48:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:17 smithi179 ceph-mon[33633]: Health check update: 12 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-79590-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: osdmap e98: 8 total, 8 up, 8 in 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1570655321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3931554385' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1570655321' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: osdmap e99: 8 total, 8 up, 8 in 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4031685142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-79590-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: osdmap e98: 8 total, 8 up, 8 in 2024-02-14T17:48:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1570655321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3931554385' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1570655321' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: osdmap e99: 8 total, 8 up, 8 in 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4031685142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi100-79590-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: osdmap e98: 8 total, 8 up, 8 in 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1570655321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3931554385' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:48:18.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-14T17:48:18.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:18.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1570655321' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:18.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: osdmap e99: 8 total, 8 up, 8 in 2024-02-14T17:48:18.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4031685142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:18.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:20.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:20.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:20.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:19 smithi179 ceph-mon[33633]: pgmap v110: 720 pgs: 296 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-02-14T17:48:20.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:20.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:20.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:20.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:19 smithi179 ceph-mon[33633]: osdmap e100: 8 total, 8 up, 8 in 2024-02-14T17:48:20.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:20.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:20.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/686626316' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-79999-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:20.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:20.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:20.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[39557]: pgmap v110: 720 pgs: 296 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-02-14T17:48:20.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:20.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:20.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:20.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[39557]: osdmap e100: 8 total, 8 up, 8 in 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/686626316' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-79999-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[30138]: pgmap v110: 720 pgs: 296 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[30138]: osdmap e100: 8 total, 8 up, 8 in 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:20.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/686626316' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-79999-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:20.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:48:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:48:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:21.285 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: Running main() from gmock_main.cc 2024-02-14T17:48:21.285 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-02-14T17:48:21.285 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [----------] Global test environment set-up. 2024-02-14T17:48:21.286 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-02-14T17:48:21.286 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-02-14T17:48:21.286 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (18235 ms) 2024-02-14T17:48:21.286 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-02-14T17:48:21.286 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (8448 ms) 2024-02-14T17:48:21.286 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-02-14T17:48:21.286 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (8239 ms) 2024-02-14T17:48:21.286 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [----------] 3 tests from NeoradosECList (34922 ms total) 2024-02-14T17:48:21.287 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: 2024-02-14T17:48:21.287 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [----------] Global test environment tear-down 2024-02-14T17:48:21.287 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (34922 ms total) 2024-02-14T17:48:21.287 INFO:tasks.workunit.client.0.smithi100.stdout: ec_list: [ PASSED ] 3 tests. 2024-02-14T17:48:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi100-80625-3"}]: dispatch 2024-02-14T17:48:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-7"}]': finished 2024-02-14T17:48:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/686626316' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-79999-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi100-80625-3"}]': finished 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1684527431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-79590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: osdmap e101: 8 total, 8 up, 8 in 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi100-80625-3"}]: dispatch 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1869281023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-79590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi100-80625-3"}]: dispatch 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-7"}]': finished 2024-02-14T17:48:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/686626316' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-79999-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:21.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi100-80625-3"}]': finished 2024-02-14T17:48:21.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1684527431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-79590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:21.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:21.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: osdmap e101: 8 total, 8 up, 8 in 2024-02-14T17:48:21.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi100-80625-3"}]: dispatch 2024-02-14T17:48:21.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1869281023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:21.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-79590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi100-80625-3"}]: dispatch 2024-02-14T17:48:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-7"}]': finished 2024-02-14T17:48:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/686626316' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi100-79999-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi100-80625-3"}]': finished 2024-02-14T17:48:21.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1684527431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-79590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:21.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:21.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: osdmap e101: 8 total, 8 up, 8 in 2024-02-14T17:48:21.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi100-80625-3"}]: dispatch 2024-02-14T17:48:21.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1869281023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:21.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-79590-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:22.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:48:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:48:22.294 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: Running main() from gmock_main.cc 2024-02-14T17:48:22.294 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-02-14T17:48:22.294 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [----------] Global test environment set-up. 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (11717 ms) 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (4388 ms) 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (4298 ms) 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (4114 ms) 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3120 ms) 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3332 ms) 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (4888 ms) 2024-02-14T17:48:22.295 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (35858 ms total) 2024-02-14T17:48:22.296 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: 2024-02-14T17:48:22.296 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [----------] Global test environment tear-down 2024-02-14T17:48:22.296 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (35858 ms total) 2024-02-14T17:48:22.296 INFO:tasks.workunit.client.0.smithi100.stdout: write_operations: [ PASSED ] 7 tests. 2024-02-14T17:48:22.397 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: pgmap v113: 680 pgs: 256 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi100-80625-3"}]': finished 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1869281023' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-79590-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: osdmap e102: 8 total, 8 up, 8 in 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2893808631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:22.398 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:22.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: pgmap v113: 680 pgs: 256 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:22.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:22.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi100-80625-3"}]': finished 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1869281023' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-79590-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: osdmap e102: 8 total, 8 up, 8 in 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2893808631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: pgmap v113: 680 pgs: 256 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 789 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:22.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1309799252' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi100-80625-3"}]': finished 2024-02-14T17:48:22.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1869281023' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi100-79612-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:22.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi100-79590-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:22.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:48:22.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: osdmap e102: 8 total, 8 up, 8 in 2024-02-14T17:48:22.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:22.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2893808631' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:22.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7"}]: dispatch 2024-02-14T17:48:22.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:22.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2893808631' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:23 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7"}]': finished 2024-02-14T17:48:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:23 smithi179 ceph-mon[33633]: osdmap e103: 8 total, 8 up, 8 in 2024-02-14T17:48:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3761939236' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:23 smithi179 ceph-mon[33633]: pgmap v116: 648 pgs: 128 unknown, 520 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-14T17:48:23.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:23.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:23 smithi179 ceph-mon[33633]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2893808631' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7"}]': finished 2024-02-14T17:48:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[39557]: osdmap e103: 8 total, 8 up, 8 in 2024-02-14T17:48:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3761939236' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[39557]: pgmap v116: 648 pgs: 128 unknown, 520 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-14T17:48:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[39557]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2893808631' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-80256-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-7"}]': finished 2024-02-14T17:48:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[30138]: osdmap e103: 8 total, 8 up, 8 in 2024-02-14T17:48:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3761939236' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[30138]: pgmap v116: 648 pgs: 128 unknown, 520 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-14T17:48:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:23 smithi100 ceph-mon[30138]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-02-14T17:48:24.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (12967 ms) 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (7626 ms) 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (4119 ms) 2024-02-14T17:48:24.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3116 ms) 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3321 ms) 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (2907 ms) 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (4014 ms) 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (38070 ms total) 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (38071 ms total) 2024-02-14T17:48:24.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-14T17:48:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3761939236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:24 smithi179 ceph-mon[33633]: osdmap e104: 8 total, 8 up, 8 in 2024-02-14T17:48:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3680968490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-79590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-79590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2956444968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-79612-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:24.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-79612-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3761939236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[30138]: osdmap e104: 8 total, 8 up, 8 in 2024-02-14T17:48:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3680968490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-79590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-79590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2956444968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-79612-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-79612-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3761939236' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[39557]: osdmap e104: 8 total, 8 up, 8 in 2024-02-14T17:48:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3680968490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-79590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-79590-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2956444968' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-79612-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-79612-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-79590-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-79612-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:25 smithi179 ceph-mon[33633]: osdmap e105: 8 total, 8 up, 8 in 2024-02-14T17:48:25.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:25.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:25.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:25 smithi179 ceph-mon[33633]: pgmap v119: 712 pgs: 256 unknown, 456 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:48:25.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-79590-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-79612-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[39557]: osdmap e105: 8 total, 8 up, 8 in 2024-02-14T17:48:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[39557]: pgmap v119: 712 pgs: 256 unknown, 456 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:48:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi100-79590-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi100-79612-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[30138]: osdmap e105: 8 total, 8 up, 8 in 2024-02-14T17:48:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[30138]: pgmap v119: 712 pgs: 256 unknown, 456 active+clean; 145 MiB data, 766 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-14T17:48:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:26.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:26.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[39557]: osdmap e106: 8 total, 8 up, 8 in 2024-02-14T17:48:26.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:26.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4"}]: dispatch 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4"}]: dispatch 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[30138]: osdmap e106: 8 total, 8 up, 8 in 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:26.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:26.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4"}]: dispatch 2024-02-14T17:48:26.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4"}]: dispatch 2024-02-14T17:48:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:26 smithi179 ceph-mon[33633]: osdmap e106: 8 total, 8 up, 8 in 2024-02-14T17:48:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:26 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:26 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4"}]: dispatch 2024-02-14T17:48:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4"}]: dispatch 2024-02-14T17:48:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4"}]': finished 2024-02-14T17:48:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: osdmap e107: 8 total, 8 up, 8 in 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/651336077' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-79612-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1250013277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-79590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-79590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-79612-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: pgmap v122: 616 pgs: 32 creating+peering, 96 unknown, 488 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-9"}]': finished 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6"}]': finished 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-79590-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-79612-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: osdmap e108: 8 total, 8 up, 8 in 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4"}]': finished 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: osdmap e107: 8 total, 8 up, 8 in 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/651336077' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-79612-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1250013277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-79590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-79590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-79612-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: pgmap v122: 616 pgs: 32 creating+peering, 96 unknown, 488 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-9"}]': finished 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6"}]': finished 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-79590-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-79612-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: osdmap e108: 8 total, 8 up, 8 in 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-9", "mode": "writeback"}]: dispatch 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi100-80161-6", "pool2": "test-rados-api-smithi100-80161-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2693891762' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-9", "mode": "writeback"}]: dispatch 2024-02-14T17:48:27.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi100-80161-6", "pool2": "test-rados-api-smithi100-80161-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-9", "mode": "writeback"}]: dispatch 2024-02-14T17:48:27.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi100-80161-6", "pool2": "test-rados-api-smithi100-80161-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2693891762' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-9", "mode": "writeback"}]: dispatch 2024-02-14T17:48:27.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi100-80161-6", "pool2": "test-rados-api-smithi100-80161-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4"}]': finished 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: osdmap e107: 8 total, 8 up, 8 in 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6"}]: dispatch 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/651336077' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-79612-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1250013277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-79590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-79590-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-79612-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: pgmap v122: 616 pgs: 32 creating+peering, 96 unknown, 488 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-9"}]': finished 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi100-80161-4", "tierpool": "test-rados-api-smithi100-80161-6"}]': finished 2024-02-14T17:48:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi100-79590-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:27.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi100-79612-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:27.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: osdmap e108: 8 total, 8 up, 8 in 2024-02-14T17:48:27.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-9", "mode": "writeback"}]: dispatch 2024-02-14T17:48:27.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2849529184' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi100-80161-6", "pool2": "test-rados-api-smithi100-80161-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2693891762' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-9", "mode": "writeback"}]: dispatch 2024-02-14T17:48:27.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi100-80161-6", "pool2": "test-rados-api-smithi100-80161-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T17:48:27.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:28 smithi179 ceph-mon[33633]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:28 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:29.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-9", "mode": "writeback"}]': finished 2024-02-14T17:48:29.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi100-80161-6", "pool2": "test-rados-api-smithi100-80161-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T17:48:29.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:29.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:28 smithi179 ceph-mon[33633]: osdmap e109: 8 total, 8 up, 8 in 2024-02-14T17:48:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[39557]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-9", "mode": "writeback"}]': finished 2024-02-14T17:48:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi100-80161-6", "pool2": "test-rados-api-smithi100-80161-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T17:48:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[39557]: osdmap e109: 8 total, 8 up, 8 in 2024-02-14T17:48:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[30138]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-9", "mode": "writeback"}]': finished 2024-02-14T17:48:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi100-80161-6", "pool2": "test-rados-api-smithi100-80161-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T17:48:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:28 smithi100 ceph-mon[30138]: osdmap e109: 8 total, 8 up, 8 in 2024-02-14T17:48:29.305 INFO:tasks.workunit.client.0.smithi100.stdout:_notify_pp: flushed 2024-02-14T17:48:29.305 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3004 ms) 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: trying... 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94818455967776 notify_id 416611827716 notifier_gid 25289 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: timed out 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: flushing 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: flushed 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3005 ms) 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: List watches 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify2 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94818455967776 notify_id 429496729605 notifier_gid 25289 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify2 done 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: watch_check 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: unwatch2 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: flushing 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: done 2024-02-14T17:48:29.306 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3005 ms) 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: List watches 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify2 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: handle_notify cookie 94818455967776 notify_id 442381631493 notifier_gid 25289 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: notify2 done 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: watch_check 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: unwatch2 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: flushing 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: done 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3007 ms) 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16335 ms total) 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (43114 ms total) 2024-02-14T17:48:29.307 INFO:tasks.workunit.client.0.smithi100.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-14T17:48:30.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:30.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:48:30.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: 13.8 deep-scrub starts 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: pgmap v125: 580 pgs: 32 creating+peering, 96 unknown, 452 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: 13.8 deep-scrub ok 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:30.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:48:30.325 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: osdmap e110: 8 total, 8 up, 8 in 2024-02-14T17:48:30.325 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:30.325 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-79925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:30.325 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4221525945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-79590-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.325 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2964862739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-79612-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.325 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-79590-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.325 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:30.325 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-79612-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.325 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:30.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:30.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:48:30.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:48:30.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: 13.8 deep-scrub starts 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: pgmap v125: 580 pgs: 32 creating+peering, 96 unknown, 452 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: 13.8 deep-scrub ok 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:48:30.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.0"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.1"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.2"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.3"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.4"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.5"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:48:30.403 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.6"}]: dispatch 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.7"}]: dispatch 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.8"}]: dispatch 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "13.9"}]: dispatch 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: 13.8 deep-scrub starts 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: pgmap v125: 580 pgs: 32 creating+peering, 96 unknown, 452 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: 13.8 deep-scrub ok 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:30.404 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: osdmap e110: 8 total, 8 up, 8 in 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-79925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4221525945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-79590-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2964862739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-79612-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-79590-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-79612-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.405 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: osdmap e110: 8 total, 8 up, 8 in 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-79925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4221525945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-79590-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2964862739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-79612-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-79590-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.406 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9"}]: dispatch 2024-02-14T17:48:30.407 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-79612-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:30.407 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:30.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:48:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:48:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.1 deep-scrub starts 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.1 deep-scrub ok 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.6 deep-scrub starts 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.6 deep-scrub ok 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.3 deep-scrub starts 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.3 deep-scrub ok 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.2 deep-scrub starts 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.2 deep-scrub ok 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.0 deep-scrub starts 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.0 deep-scrub ok 2024-02-14T17:48:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.4 deep-scrub starts 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.4 deep-scrub ok 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.5 deep-scrub starts 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.5 deep-scrub ok 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-79590-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9"}]': finished 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-79612-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.9 deep-scrub starts 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: 13.9 deep-scrub ok 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: osdmap e111: 8 total, 8 up, 8 in 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: pgmap v128: 644 pgs: 224 unknown, 420 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:31.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.1 deep-scrub starts 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.1 deep-scrub ok 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.6 deep-scrub starts 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.6 deep-scrub ok 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.3 deep-scrub starts 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.3 deep-scrub ok 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.2 deep-scrub starts 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.2 deep-scrub ok 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.0 deep-scrub starts 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.0 deep-scrub ok 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.4 deep-scrub starts 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.4 deep-scrub ok 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.5 deep-scrub starts 2024-02-14T17:48:31.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.5 deep-scrub ok 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-79590-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9"}]': finished 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-79612-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.9 deep-scrub starts 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: 13.9 deep-scrub ok 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: osdmap e111: 8 total, 8 up, 8 in 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: pgmap v128: 644 pgs: 224 unknown, 420 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.1 deep-scrub starts 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.1 deep-scrub ok 2024-02-14T17:48:31.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.6 deep-scrub starts 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.6 deep-scrub ok 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.3 deep-scrub starts 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.3 deep-scrub ok 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.2 deep-scrub starts 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.2 deep-scrub ok 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.0 deep-scrub starts 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.0 deep-scrub ok 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.4 deep-scrub starts 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.4 deep-scrub ok 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.5 deep-scrub starts 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.5 deep-scrub ok 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi100-79590-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-9"}]': finished 2024-02-14T17:48:31.403 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi100-79612-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:31.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.9 deep-scrub starts 2024-02-14T17:48:31.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: 13.9 deep-scrub ok 2024-02-14T17:48:31.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: osdmap e111: 8 total, 8 up, 8 in 2024-02-14T17:48:31.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: pgmap v128: 644 pgs: 224 unknown, 420 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:31.404 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:32.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:48:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:48:32.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:32 smithi179 ceph-mon[33633]: 13.7 deep-scrub starts 2024-02-14T17:48:32.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:32 smithi179 ceph-mon[33633]: 13.7 deep-scrub ok 2024-02-14T17:48:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-79925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]': finished 2024-02-14T17:48:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:32 smithi179 ceph-mon[33633]: osdmap e112: 8 total, 8 up, 8 in 2024-02-14T17:48:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:32 smithi100 ceph-mon[30138]: 13.7 deep-scrub starts 2024-02-14T17:48:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:32 smithi100 ceph-mon[30138]: 13.7 deep-scrub ok 2024-02-14T17:48:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-79925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]': finished 2024-02-14T17:48:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:32 smithi100 ceph-mon[30138]: osdmap e112: 8 total, 8 up, 8 in 2024-02-14T17:48:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:32.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:32 smithi100 ceph-mon[39557]: 13.7 deep-scrub starts 2024-02-14T17:48:32.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:32 smithi100 ceph-mon[39557]: 13.7 deep-scrub ok 2024-02-14T17:48:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi100-79925-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]': finished 2024-02-14T17:48:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:32 smithi100 ceph-mon[39557]: osdmap e112: 8 total, 8 up, 8 in 2024-02-14T17:48:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:33 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1804927147' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:33 smithi179 ceph-mon[33633]: pgmap v131: 556 pgs: 32 creating+peering, 136 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-14T17:48:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:33 smithi179 ceph-mon[33633]: osdmap e113: 8 total, 8 up, 8 in 2024-02-14T17:48:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1804927147' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[30138]: pgmap v131: 556 pgs: 32 creating+peering, 136 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-14T17:48:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[30138]: osdmap e113: 8 total, 8 up, 8 in 2024-02-14T17:48:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1804927147' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[39557]: pgmap v131: 556 pgs: 32 creating+peering, 136 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-14T17:48:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[39557]: osdmap e113: 8 total, 8 up, 8 in 2024-02-14T17:48:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:34.327 INFO:tasks.workunit.client.0.smithi100.stdout: misc: Running main() from gmock_main.cc 2024-02-14T17:48:34.327 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-02-14T17:48:34.327 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [----------] Global test environment set-up. 2024-02-14T17:48:34.327 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-02-14T17:48:34.327 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-02-14T17:48:34.327 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.Version (9408 ms) 2024-02-14T17:48:34.327 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (5967 ms) 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.LongName (5060 ms) 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (4111 ms) 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (3101 ms) 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3349 ms) 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.Exec (2884 ms) 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3026 ms) 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-02-14T17:48:34.328 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3006 ms) 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.BigObject (2970 ms) 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2007 ms) 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3019 ms) 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [----------] 12 tests from NeoRadosMisc (47910 ms total) 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [----------] Global test environment tear-down 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [==========] 12 tests from 1 test suite ran. (47910 ms total) 2024-02-14T17:48:34.329 INFO:tasks.workunit.client.0.smithi100.stdout: misc: [ PASSED ] 12 tests. 2024-02-14T17:48:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:34 smithi179 ceph-mon[33633]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:34.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:34.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:34 smithi179 ceph-mon[33633]: osdmap e114: 8 total, 8 up, 8 in 2024-02-14T17:48:34.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3627591093' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-79590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-79590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[39557]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[39557]: osdmap e114: 8 total, 8 up, 8 in 2024-02-14T17:48:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3627591093' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-79590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-79590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[30138]: Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[30138]: osdmap e114: 8 total, 8 up, 8 in 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3627591093' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-79590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-79590-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:34.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:35 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:35.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:35 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-79590-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:35.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:35 smithi179 ceph-mon[33633]: osdmap e115: 8 total, 8 up, 8 in 2024-02-14T17:48:35.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:35.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:35 smithi179 ceph-mon[33633]: pgmap v134: 556 pgs: 32 creating+peering, 136 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 7 op/s 2024-02-14T17:48:35.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:35.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:35 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-79590-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[39557]: osdmap e115: 8 total, 8 up, 8 in 2024-02-14T17:48:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[39557]: pgmap v134: 556 pgs: 32 creating+peering, 136 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 7 op/s 2024-02-14T17:48:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:35.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:35.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79972-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:35.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi100-79590-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:35.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[30138]: osdmap e115: 8 total, 8 up, 8 in 2024-02-14T17:48:35.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:35.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[30138]: pgmap v134: 556 pgs: 32 creating+peering, 136 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 18 MiB/s wr, 7 op/s 2024-02-14T17:48:35.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:35.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:35 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[30138]: osdmap e116: 8 total, 8 up, 8 in 2024-02-14T17:48:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:48:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/642118347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-79612-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:48:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-79612-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[39557]: osdmap e116: 8 total, 8 up, 8 in 2024-02-14T17:48:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:48:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/642118347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-79612-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:48:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-79612-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:36 smithi179 ceph-mon[33633]: osdmap e116: 8 total, 8 up, 8 in 2024-02-14T17:48:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:48:36.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/642118347' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-79612-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:36.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache", "force_nonempty":""}]: dispatch 2024-02-14T17:48:36.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-79612-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:36.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:36.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache", "force_nonempty":""}]': finished 2024-02-14T17:48:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-79612-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-11"}]': finished 2024-02-14T17:48:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-11", "mode": "writeback"}]: dispatch 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: osdmap e117: 8 total, 8 up, 8 in 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79972-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-11", "mode": "writeback"}]: dispatch 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79972-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/916541060' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-79590-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: pgmap v137: 620 pgs: 64 unknown, 96 creating+peering, 460 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-11", "mode": "writeback"}]': finished 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79972-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/916541060' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-79590-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: osdmap e118: 8 total, 8 up, 8 in 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache"}]: dispatch 2024-02-14T17:48:37.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache"}]: dispatch 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache", "force_nonempty":""}]': finished 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-79612-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-11"}]': finished 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-11", "mode": "writeback"}]: dispatch 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: osdmap e117: 8 total, 8 up, 8 in 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79972-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-11", "mode": "writeback"}]: dispatch 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79972-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/916541060' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-79590-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: pgmap v137: 620 pgs: 64 unknown, 96 creating+peering, 460 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-11", "mode": "writeback"}]': finished 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79972-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:37.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/916541060' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-79590-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:37.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: osdmap e118: 8 total, 8 up, 8 in 2024-02-14T17:48:37.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache"}]: dispatch 2024-02-14T17:48:37.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache"}]: dispatch 2024-02-14T17:48:37.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache", "force_nonempty":""}]': finished 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi100-79612-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-11"}]': finished 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-11", "mode": "writeback"}]: dispatch 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: osdmap e117: 8 total, 8 up, 8 in 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79972-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-11", "mode": "writeback"}]: dispatch 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79972-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/916541060' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-79590-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: pgmap v137: 620 pgs: 64 unknown, 96 creating+peering, 460 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-11", "mode": "writeback"}]': finished 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79972-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/916541060' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi100-79590-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: osdmap e118: 8 total, 8 up, 8 in 2024-02-14T17:48:37.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1989208584' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache"}]: dispatch 2024-02-14T17:48:37.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache"}]: dispatch 2024-02-14T17:48:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache"}]': finished 2024-02-14T17:48:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:38 smithi179 ceph-mon[33633]: osdmap e119: 8 total, 8 up, 8 in 2024-02-14T17:48:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1018720188' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-79612-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-79612-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache"}]': finished 2024-02-14T17:48:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:38 smithi100 ceph-mon[39557]: osdmap e119: 8 total, 8 up, 8 in 2024-02-14T17:48:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1018720188' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-79612-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-79612-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79972-10", "tierpool":"test-rados-api-smithi100-79972-10-cache"}]': finished 2024-02-14T17:48:39.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:38 smithi100 ceph-mon[30138]: osdmap e119: 8 total, 8 up, 8 in 2024-02-14T17:48:39.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1018720188' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-79612-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:39.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-79612-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:40.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: pgmap v140: 588 pgs: 96 unknown, 32 creating+peering, 460 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-79612-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: osdmap e120: 8 total, 8 up, 8 in 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3519917908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-79590-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-79590-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:40.053 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: pgmap v140: 588 pgs: 96 unknown, 32 creating+peering, 460 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-02-14T17:48:40.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:40.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:40.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-79612-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:40.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:48:40.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: osdmap e120: 8 total, 8 up, 8 in 2024-02-14T17:48:40.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:40.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3519917908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-79590-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:40.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:40.054 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-79590-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: pgmap v140: 588 pgs: 96 unknown, 32 creating+peering, 460 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 259 KiB/s wr, 5 op/s 2024-02-14T17:48:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:48:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi100-79612-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:48:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: osdmap e120: 8 total, 8 up, 8 in 2024-02-14T17:48:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:40.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3519917908' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-79590-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:40.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11"}]: dispatch 2024-02-14T17:48:40.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-79590-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:40.268 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: Running main() from gmock_main.cc 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [----------] Global test environment set-up. 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolList (11171 ms) 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (7806 ms) 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3775 ms) 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (4198 ms) 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3316 ms) 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (4911 ms) 2024-02-14T17:48:40.269 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-14T17:48:40.270 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5037 ms) 2024-02-14T17:48:40.270 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-14T17:48:40.270 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4977 ms) 2024-02-14T17:48:40.270 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-14T17:48:40.270 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8957 ms) 2024-02-14T17:48:40.270 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [----------] 9 tests from LibRadosPools (54148 ms total) 2024-02-14T17:48:40.270 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: 2024-02-14T17:48:40.270 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [----------] Global test environment tear-down 2024-02-14T17:48:40.270 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (54148 ms total) 2024-02-14T17:48:40.270 INFO:tasks.workunit.client.0.smithi100.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-14T17:48:40.814 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:48:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:41.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:40 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11"}]': finished 2024-02-14T17:48:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-79590-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:40 smithi179 ceph-mon[33633]: osdmap e121: 8 total, 8 up, 8 in 2024-02-14T17:48:41.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:41.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:41.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11"}]': finished 2024-02-14T17:48:41.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-79590-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:41.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-mon[30138]: osdmap e121: 8 total, 8 up, 8 in 2024-02-14T17:48:41.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:41.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:48:41.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-11"}]': finished 2024-02-14T17:48:41.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi100-79590-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:41.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:40 smithi100 ceph-mon[39557]: osdmap e121: 8 total, 8 up, 8 in 2024-02-14T17:48:42.004 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:48:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:48:42.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:42 smithi179 ceph-mon[33633]: pgmap v143: 524 pgs: 96 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:42.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:42.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:42 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:42.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:42.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:42 smithi179 ceph-mon[33633]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:42.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:42 smithi179 ceph-mon[33633]: osdmap e122: 8 total, 8 up, 8 in 2024-02-14T17:48:42.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2995848402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-79612-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:42.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:42.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-79612-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[30138]: pgmap v143: 524 pgs: 96 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[30138]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[30138]: osdmap e122: 8 total, 8 up, 8 in 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2995848402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-79612-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-79612-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[39557]: pgmap v143: 524 pgs: 96 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[39557]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[39557]: osdmap e122: 8 total, 8 up, 8 in 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2995848402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-79612-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-79612-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.283 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: Running main() from gmock_main.cc 2024-02-14T17:48:43.283 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-02-14T17:48:43.283 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [----------] Global test environment set-up. 2024-02-14T17:48:43.283 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-02-14T17:48:43.283 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-02-14T17:48:43.283 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (10011 ms) 2024-02-14T17:48:43.283 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-02-14T17:48:43.283 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (7400 ms) 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (5045 ms) 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3082 ms) 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (3439 ms) 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3861 ms) 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3053 ms) 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (3010 ms) 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-02-14T17:48:43.284 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (2932 ms) 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (3053 ms) 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3013 ms) 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3013 ms) 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (2936 ms) 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3009 ms) 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (56857 ms total) 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [----------] Global test environment tear-down 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (56857 ms total) 2024-02-14T17:48:43.285 INFO:tasks.workunit.client.0.smithi100.stdout: read_operations: [ PASSED ] 14 tests. 2024-02-14T17:48:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]': finished 2024-02-14T17:48:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-79612-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: osdmap e123: 8 total, 8 up, 8 in 2024-02-14T17:48:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1373177526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-79590-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-79590-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: pgmap v146: 580 pgs: 160 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-02-14T17:48:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]': finished 2024-02-14T17:48:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-79612-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: osdmap e123: 8 total, 8 up, 8 in 2024-02-14T17:48:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1373177526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-79590-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-79590-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: pgmap v146: 580 pgs: 160 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi100-79925-10"}]': finished 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi100-79612-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: osdmap e123: 8 total, 8 up, 8 in 2024-02-14T17:48:43.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-79925-10"}]: dispatch 2024-02-14T17:48:43.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1373177526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-79590-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-79590-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:43.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: pgmap v146: 580 pgs: 160 unknown, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-02-14T17:48:43.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:44.296 INFO:tasks.workunit.client.0.smithi100.stdout: io: Running main() from gmock_main.cc 2024-02-14T17:48:44.296 INFO:tasks.workunit.client.0.smithi100.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-02-14T17:48:44.296 INFO:tasks.workunit.client.0.smithi100.stdout: io: [----------] Global test environment set-up. 2024-02-14T17:48:44.296 INFO:tasks.workunit.client.0.smithi100.stdout: io: [----------] 14 tests from NeoRadosIo 2024-02-14T17:48:44.296 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.Limits (12865 ms) 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (5128 ms) 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.ReadOp (4548 ms) 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.SparseRead (3082 ms) 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3443 ms) 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (3861 ms) 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (4050 ms) 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3017 ms) 2024-02-14T17:48:44.297 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (2969 ms) 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.Trunc (3507 ms) 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.Remove (2532 ms) 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (2928 ms) 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.RmXattr (3009 ms) 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3019 ms) 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [----------] 14 tests from NeoRadosIo (57958 ms total) 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: 2024-02-14T17:48:44.298 INFO:tasks.workunit.client.0.smithi100.stdout: io: [----------] Global test environment tear-down 2024-02-14T17:48:44.299 INFO:tasks.workunit.client.0.smithi100.stdout: io: [==========] 14 tests from 1 test suite ran. (57958 ms total) 2024-02-14T17:48:44.299 INFO:tasks.workunit.client.0.smithi100.stdout: io: [ PASSED ] 14 tests. 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-79925-10"}]': finished 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-79590-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: osdmap e124: 8 total, 8 up, 8 in 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-79925-10"}]': finished 2024-02-14T17:48:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-79590-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: osdmap e124: 8 total, 8 up, 8 in 2024-02-14T17:48:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3312536088' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi100-79925-10"}]': finished 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi100-79590-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: osdmap e124: 8 total, 8 up, 8 in 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:44.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:48:44.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:45 smithi179 ceph-mon[33633]: osdmap e125: 8 total, 8 up, 8 in 2024-02-14T17:48:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3118323988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-79612-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:45.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:45 smithi179 ceph-mon[33633]: pgmap v149: 452 pgs: 64 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:48:45.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[39557]: osdmap e125: 8 total, 8 up, 8 in 2024-02-14T17:48:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3118323988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-79612-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[39557]: pgmap v149: 452 pgs: 64 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:48:45.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:48:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[30138]: osdmap e125: 8 total, 8 up, 8 in 2024-02-14T17:48:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3118323988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-79612-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[30138]: pgmap v149: 452 pgs: 64 unknown, 388 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:48:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3118323988' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-79612-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:46 smithi179 ceph-mon[33633]: osdmap e126: 8 total, 8 up, 8 in 2024-02-14T17:48:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3675528907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-79590-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-79590-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:46.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3118323988' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-79612-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:46.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:46 smithi100 ceph-mon[30138]: osdmap e126: 8 total, 8 up, 8 in 2024-02-14T17:48:46.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3675528907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-79590-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:46.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-79590-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:46.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:46.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3118323988' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi100-79612-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:46.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:46 smithi100 ceph-mon[39557]: osdmap e126: 8 total, 8 up, 8 in 2024-02-14T17:48:46.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3675528907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-79590-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:46.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-79590-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:46.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]': finished 2024-02-14T17:48:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-79590-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:47 smithi179 ceph-mon[33633]: osdmap e127: 8 total, 8 up, 8 in 2024-02-14T17:48:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:47 smithi179 ceph-mon[33633]: pgmap v152: 492 pgs: 64 creating+peering, 8 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 72 op/s 2024-02-14T17:48:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]': finished 2024-02-14T17:48:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-79590-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:47 smithi100 ceph-mon[39557]: osdmap e127: 8 total, 8 up, 8 in 2024-02-14T17:48:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:47 smithi100 ceph-mon[39557]: pgmap v152: 492 pgs: 64 creating+peering, 8 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 72 op/s 2024-02-14T17:48:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi100-79925-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]': finished 2024-02-14T17:48:47.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi100-79590-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:47.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:47 smithi100 ceph-mon[30138]: osdmap e127: 8 total, 8 up, 8 in 2024-02-14T17:48:47.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:47 smithi100 ceph-mon[30138]: pgmap v152: 492 pgs: 64 creating+peering, 8 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 72 op/s 2024-02-14T17:48:47.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:48.438 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-14T17:48:48.438 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] Global test environment set-up. 2024-02-14T17:48:48.438 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-14T17:48:48.438 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-14T17:48:48.438 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-14T17:48:48.438 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-14T17:48:48.438 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-02-14T17:48:48.438 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-14T17:48:48.438 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-02-14T17:47:46.872+0000 7f8abc446780 -1 failed for service _ceph-mon._tcp 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-02-14T17:47:46.872+0000 7f8abc446780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (798 ms) 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2007 ms) 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2805 ms total) 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4421600 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47669e0 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4767380 2024-02-14T17:48:48.439 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4766e90 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4767810 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4767cb0 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47687a0 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4768560 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4768c10 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4768670 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47691f0 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4768d50 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4768e60 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4768f70 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4769320 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4769470 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4769a20 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4769580 2024-02-14T17:48:48.440 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4769690 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47697a0 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47698d0 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4769fd0 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4769b30 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4769c40 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4769d50 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4769ec0 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476a5c0 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476a120 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476a250 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476a360 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476a490 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476ab70 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476a6d0 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476a7e0 2024-02-14T17:48:48.441 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476a930 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476aa60 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476b160 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476acc0 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476add0 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476aee0 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476aff0 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476b750 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476b2b0 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476b3e0 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476b4f0 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476b860 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476b970 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476ba80 2024-02-14T17:48:48.442 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476bb90 2024-02-14T17:48:48.443 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476c1e0 2024-02-14T17:48:48.443 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476bd40 2024-02-14T17:48:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: osdmap e128: 8 total, 8 up, 8 in 2024-02-14T17:48:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3876373711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-79612-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3876373711' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-79612-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: osdmap e129: 8 total, 8 up, 8 in 2024-02-14T17:48:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:48:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:48:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3196186124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi100-79590-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: osdmap e128: 8 total, 8 up, 8 in 2024-02-14T17:48:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3876373711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-79612-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3876373711' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-79612-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: osdmap e129: 8 total, 8 up, 8 in 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3196186124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi100-79590-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: osdmap e128: 8 total, 8 up, 8 in 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3876373711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-79612-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:48.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:48:48.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:48.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3876373711' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi100-79612-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:48.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:48:48.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: osdmap e129: 8 total, 8 up, 8 in 2024-02-14T17:48:48.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:48:48.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:48:48.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3196186124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi100-79590-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:49.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:49 smithi179 ceph-mon[33633]: pgmap v155: 524 pgs: 32 creating+peering, 72 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-02-14T17:48:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:49 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-13"}]': finished 2024-02-14T17:48:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3196186124' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi100-79590-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:49 smithi179 ceph-mon[33633]: osdmap e130: 8 total, 8 up, 8 in 2024-02-14T17:48:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-13", "mode": "writeback"}]: dispatch 2024-02-14T17:48:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:49 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-13", "mode": "writeback"}]: dispatch 2024-02-14T17:48:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[39557]: pgmap v155: 524 pgs: 32 creating+peering, 72 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-02-14T17:48:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-13"}]': finished 2024-02-14T17:48:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3196186124' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi100-79590-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[39557]: osdmap e130: 8 total, 8 up, 8 in 2024-02-14T17:48:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-13", "mode": "writeback"}]: dispatch 2024-02-14T17:48:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-13", "mode": "writeback"}]: dispatch 2024-02-14T17:48:49.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[30138]: pgmap v155: 524 pgs: 32 creating+peering, 72 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-02-14T17:48:49.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:49.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-13"}]': finished 2024-02-14T17:48:49.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3196186124' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi100-79590-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:49.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[30138]: osdmap e130: 8 total, 8 up, 8 in 2024-02-14T17:48:49.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-13", "mode": "writeback"}]: dispatch 2024-02-14T17:48:49.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:49 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-13", "mode": "writeback"}]: dispatch 2024-02-14T17:48:50.808 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:48:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:48:51.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:51.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:50 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:51.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-13", "mode": "writeback"}]': finished 2024-02-14T17:48:51.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:50 smithi179 ceph-mon[33633]: osdmap e131: 8 total, 8 up, 8 in 2024-02-14T17:48:51.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1311766932' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-79612-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:51.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-79612-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:51.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:51.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:51.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-13", "mode": "writeback"}]': finished 2024-02-14T17:48:51.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[30138]: osdmap e131: 8 total, 8 up, 8 in 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1311766932' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-79612-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-79612-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-13", "mode": "writeback"}]': finished 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[39557]: osdmap e131: 8 total, 8 up, 8 in 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1311766932' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-79612-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:51.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-79612-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:52.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:48:52.010 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: pgmap v158: 460 pgs: 40 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:52.010 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:52.010 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:48:52.010 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:48:52.011 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: 170.0 scrub starts 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: 170.0 scrub ok 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: 170.4 scrub starts 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: 170.4 scrub ok 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: 170.9 scrub starts 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: 170.9 scrub ok 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-79612-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: osdmap e132: 8 total, 8 up, 8 in 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/404448790' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-79590-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:52.012 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-79590-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:52.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: pgmap v158: 460 pgs: 40 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:52.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:52.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:48:52.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:48:52.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:48:52.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:48:52.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:48:52.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: 170.0 scrub starts 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: 170.0 scrub ok 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: 170.4 scrub starts 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: 170.4 scrub ok 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: 170.9 scrub starts 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: 170.9 scrub ok 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-79612-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: osdmap e132: 8 total, 8 up, 8 in 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/404448790' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-79590-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-79590-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:52.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: pgmap v158: 460 pgs: 40 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.0"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.1"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.2"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.3"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.4"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.5"}]: dispatch 2024-02-14T17:48:52.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.6"}]: dispatch 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.7"}]: dispatch 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.8"}]: dispatch 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "170.9"}]: dispatch 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: 170.0 scrub starts 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: 170.0 scrub ok 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: 170.4 scrub starts 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: 170.4 scrub ok 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: 170.9 scrub starts 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: 170.9 scrub ok 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi100-79612-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:52.155 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: osdmap e132: 8 total, 8 up, 8 in 2024-02-14T17:48:52.156 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/404448790' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-79590-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:52.156 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-79590-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: 170.3 scrub starts 2024-02-14T17:48:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: 170.3 scrub ok 2024-02-14T17:48:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: 170.2 scrub starts 2024-02-14T17:48:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: 170.2 scrub ok 2024-02-14T17:48:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:53.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:53.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: 170.5 scrub starts 2024-02-14T17:48:53.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: 170.5 scrub ok 2024-02-14T17:48:53.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: 170.6 scrub starts 2024-02-14T17:48:53.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: 170.6 scrub ok 2024-02-14T17:48:53.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: 170.8 scrub starts 2024-02-14T17:48:53.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: 170.8 scrub ok 2024-02-14T17:48:53.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-79590-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:53.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:52 smithi179 ceph-mon[33633]: osdmap e133: 8 total, 8 up, 8 in 2024-02-14T17:48:53.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: 170.3 scrub starts 2024-02-14T17:48:53.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: 170.3 scrub ok 2024-02-14T17:48:53.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: 170.2 scrub starts 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: 170.2 scrub ok 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: 170.5 scrub starts 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: 170.5 scrub ok 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: 170.6 scrub starts 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: 170.6 scrub ok 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: 170.8 scrub starts 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: 170.8 scrub ok 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-79590-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[30138]: osdmap e133: 8 total, 8 up, 8 in 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: 170.3 scrub starts 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: 170.3 scrub ok 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: 170.2 scrub starts 2024-02-14T17:48:53.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: 170.2 scrub ok 2024-02-14T17:48:53.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:48:53.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:53.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: 170.5 scrub starts 2024-02-14T17:48:53.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: 170.5 scrub ok 2024-02-14T17:48:53.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: 170.6 scrub starts 2024-02-14T17:48:53.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: 170.6 scrub ok 2024-02-14T17:48:53.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: 170.8 scrub starts 2024-02-14T17:48:53.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: 170.8 scrub ok 2024-02-14T17:48:53.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi100-79590-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:53.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:52 smithi100 ceph-mon[39557]: osdmap e133: 8 total, 8 up, 8 in 2024-02-14T17:48:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:53 smithi179 ceph-mon[33633]: pgmap v161: 492 pgs: 6 creating+peering, 58 unknown, 428 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-14T17:48:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:53 smithi179 ceph-mon[33633]: 170.1 deep-scrub starts 2024-02-14T17:48:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:53 smithi179 ceph-mon[33633]: 170.7 scrub starts 2024-02-14T17:48:54.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:53 smithi179 ceph-mon[33633]: 170.1 deep-scrub ok 2024-02-14T17:48:54.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:53 smithi179 ceph-mon[33633]: 170.7 scrub ok 2024-02-14T17:48:54.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:53 smithi179 ceph-mon[33633]: osdmap e134: 8 total, 8 up, 8 in 2024-02-14T17:48:54.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1855117145' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-79612-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:54.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-79612-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[39557]: pgmap v161: 492 pgs: 6 creating+peering, 58 unknown, 428 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-14T17:48:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[39557]: 170.1 deep-scrub starts 2024-02-14T17:48:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[39557]: 170.7 scrub starts 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[39557]: 170.1 deep-scrub ok 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[39557]: 170.7 scrub ok 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[39557]: osdmap e134: 8 total, 8 up, 8 in 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1855117145' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-79612-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-79612-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[30138]: pgmap v161: 492 pgs: 6 creating+peering, 58 unknown, 428 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[30138]: 170.1 deep-scrub starts 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[30138]: 170.7 scrub starts 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[30138]: 170.1 deep-scrub ok 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[30138]: 170.7 scrub ok 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[30138]: osdmap e134: 8 total, 8 up, 8 in 2024-02-14T17:48:54.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1855117145' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-79612-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:54.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-79612-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-79612-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:54 smithi179 ceph-mon[33633]: osdmap e135: 8 total, 8 up, 8 in 2024-02-14T17:48:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3880456040' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-79590-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:55.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:55.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-79612-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:55.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:54 smithi100 ceph-mon[30138]: osdmap e135: 8 total, 8 up, 8 in 2024-02-14T17:48:55.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3880456040' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-79590-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:55.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:55.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi100-79612-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:55.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:54 smithi100 ceph-mon[39557]: osdmap e135: 8 total, 8 up, 8 in 2024-02-14T17:48:55.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3880456040' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-79590-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:56.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:55 smithi179 ceph-mon[33633]: pgmap v164: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-14T17:48:56.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:56.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3880456040' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-79590-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:56.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:55 smithi179 ceph-mon[33633]: osdmap e136: 8 total, 8 up, 8 in 2024-02-14T17:48:56.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:56.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:56.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[30138]: pgmap v164: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-14T17:48:56.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:56.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3880456040' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-79590-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:56.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[30138]: osdmap e136: 8 total, 8 up, 8 in 2024-02-14T17:48:56.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:56.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:56.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[39557]: pgmap v164: 492 pgs: 64 unknown, 428 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-14T17:48:56.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:56.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3880456040' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi100-79590-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:56.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[39557]: osdmap e136: 8 total, 8 up, 8 in 2024-02-14T17:48:56.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:56.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:56 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:56 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]': finished 2024-02-14T17:48:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:56 smithi179 ceph-mon[33633]: osdmap e137: 8 total, 8 up, 8 in 2024-02-14T17:48:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/641528289' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]': finished 2024-02-14T17:48:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[30138]: osdmap e137: 8 total, 8 up, 8 in 2024-02-14T17:48:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/641528289' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:57.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:57.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:48:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:48:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]': finished 2024-02-14T17:48:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[39557]: osdmap e137: 8 total, 8 up, 8 in 2024-02-14T17:48:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2633948173' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/641528289' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:57.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]: dispatch 2024-02-14T17:48:57.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: starte api_snapshots: Running main() from gmock_main.cc 2024-02-14T17:48:57.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-02-14T17:48:57.312 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (4382 ms) 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (3341 ms) 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2558 ms) 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (3027 ms) 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (13308 ms total) 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (5529 ms) 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-14T17:48:57.313 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4884 ms) 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (5024 ms) 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (15438 ms total) 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2729 ms) 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2011 ms) 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1938 ms) 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1994 ms) 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8672 ms total) 2024-02-14T17:48:57.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: 2024-02-14T17:48:57.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-14T17:48:57.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-14T17:48:57.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (3995 ms) 2024-02-14T17:48:57.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-14T17:48:57.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4019 ms) 2024-02-14T17:48:57.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8014 ms total) 2024-02-14T17:48:57.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: 2024-02-14T17:48:57.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-14T17:48:57.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (71205 ms total) 2024-02-14T17:48:57.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-02-14T17:48:58.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[30138]: pgmap v167: 484 pgs: 32 creating+peering, 32 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-02-14T17:48:58.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:58.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]': finished 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[30138]: osdmap e138: 8 total, 8 up, 8 in 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1689142993' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-79590-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-79590-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[39557]: pgmap v167: 484 pgs: 32 creating+peering, 32 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]': finished 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[39557]: osdmap e138: 8 total, 8 up, 8 in 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1689142993' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-79590-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:58.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-79590-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:57 smithi179 ceph-mon[33633]: pgmap v167: 484 pgs: 32 creating+peering, 32 unknown, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-02-14T17:48:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:58.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi100-79925-15"}]': finished 2024-02-14T17:48:58.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:57 smithi179 ceph-mon[33633]: osdmap e138: 8 total, 8 up, 8 in 2024-02-14T17:48:58.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1689142993' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-79590-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:58.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-79590-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:48:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-79590-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:58 smithi100 ceph-mon[39557]: osdmap e139: 8 total, 8 up, 8 in 2024-02-14T17:48:59.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:59.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-79590-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:59.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:58 smithi100 ceph-mon[30138]: osdmap e139: 8 total, 8 up, 8 in 2024-02-14T17:48:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:48:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi100-79590-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:48:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:58 smithi179 ceph-mon[33633]: osdmap e139: 8 total, 8 up, 8 in 2024-02-14T17:49:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-14T17:49:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: pgmap v170: 452 pgs: 32 creating+peering, 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:49:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: osdmap e140: 8 total, 8 up, 8 in 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2363227131' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-79612-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2"}]: dispatch 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-79612-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3062361267' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79999-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79999-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: pgmap v170: 452 pgs: 32 creating+peering, 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: osdmap e140: 8 total, 8 up, 8 in 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2363227131' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-79612-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2"}]: dispatch 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-79612-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3062361267' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79999-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:48:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79999-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-14T17:49:00.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: pgmap v170: 452 pgs: 32 creating+peering, 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:49:00.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:00.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:00.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:00.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: osdmap e140: 8 total, 8 up, 8 in 2024-02-14T17:49:00.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2363227131' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-79612-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2"}]: dispatch 2024-02-14T17:49:00.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-79612-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3062361267' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79999-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:48:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79999-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:00.831 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:49:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-79612-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79999-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[39557]: osdmap e141: 8 total, 8 up, 8 in 2024-02-14T17:49:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/321013050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-79590-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-79612-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79999-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[30138]: osdmap e141: 8 total, 8 up, 8 in 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:49:01.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/321013050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-79590-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:01.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi100-79612-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:01.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi100-79749-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:01.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79999-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:01.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:00 smithi179 ceph-mon[33633]: osdmap e141: 8 total, 8 up, 8 in 2024-02-14T17:49:01.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:49:01.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:49:01.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T17:49:01.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/321013050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-79590-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:02.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:49:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:49:02.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:01 smithi179 ceph-mon[33633]: pgmap v173: 516 pgs: 32 creating+peering, 96 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:49:02.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:02.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T17:49:02.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/321013050' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-79590-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:02.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:01 smithi179 ceph-mon[33633]: osdmap e142: 8 total, 8 up, 8 in 2024-02-14T17:49:02.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:49:02.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:49:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[39557]: pgmap v173: 516 pgs: 32 creating+peering, 96 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:49:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T17:49:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/321013050' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-79590-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[39557]: osdmap e142: 8 total, 8 up, 8 in 2024-02-14T17:49:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:49:02.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:49:02.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[30138]: pgmap v173: 516 pgs: 32 creating+peering, 96 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:49:02.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:02.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T17:49:02.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/321013050' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi100-79590-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:02.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[30138]: osdmap e142: 8 total, 8 up, 8 in 2024-02-14T17:49:02.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:49:02.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T17:49:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T17:49:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[39557]: osdmap e143: 8 total, 8 up, 8 in 2024-02-14T17:49:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/638819138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-79612-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:49:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-79612-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:49:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:03.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T17:49:03.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[30138]: osdmap e143: 8 total, 8 up, 8 in 2024-02-14T17:49:03.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/638819138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-79612-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:49:03.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-79612-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:49:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:02 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T17:49:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:02 smithi179 ceph-mon[33633]: osdmap e143: 8 total, 8 up, 8 in 2024-02-14T17:49:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/638819138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-79612-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4066943880' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:49:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-79612-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T17:49:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[39557]: pgmap v176: 484 pgs: 32 unknown, 452 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:49:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-79612-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1"}]': finished 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[39557]: osdmap e144: 8 total, 8 up, 8 in 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2435037412' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79590-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[30138]: pgmap v176: 484 pgs: 32 unknown, 452 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-79612-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1"}]': finished 2024-02-14T17:49:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[30138]: osdmap e144: 8 total, 8 up, 8 in 2024-02-14T17:49:04.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2435037412' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79590-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:04.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:04.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:04.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:03 smithi179 ceph-mon[33633]: pgmap v176: 484 pgs: 32 unknown, 452 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:49:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi100-79612-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi100-79749-1","app":"app1","key":"key1"}]': finished 2024-02-14T17:49:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:03 smithi179 ceph-mon[33633]: osdmap e144: 8 total, 8 up, 8 in 2024-02-14T17:49:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2435037412' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79590-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:04.329 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-14T17:49:04.329 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-14T17:49:04.329 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-14T17:49:04.329 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-14T17:49:04.329 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-14T17:49:04.329 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-14T17:49:04.329 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-02-14T17:49:04.329 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: seed 79749 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (1 ms) 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (116 ms) 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (1204 ms) 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (18 ms) 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (10 ms) 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-14T17:49:04.330 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (5 ms) 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (11 ms) 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (5 ms) 2024-02-14T17:49:04.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (9 ms) 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1524 ms) 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (307 ms) 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: done waiting, doing copies 2024-02-14T17:49:04.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: done waiting 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60691 ms) 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (3 ms) 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4591 ms) 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-14T17:49:04.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-14T17:49:04.334 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-14T17:49:05.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:05.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2435037412' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79590-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: osdmap e145: 8 total, 8 up, 8 in 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-79999-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79749-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79749-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2435037412' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79590-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: osdmap e145: 8 total, 8 up, 8 in 2024-02-14T17:49:05.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-79999-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:05.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79749-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:05.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79749-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2435037412' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79590-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: osdmap e145: 8 total, 8 up, 8 in 2024-02-14T17:49:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-79999-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79749-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79749-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[30138]: pgmap v179: 420 pgs: 32 unknown, 388 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:49:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79749-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79749-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[30138]: osdmap e146: 8 total, 8 up, 8 in 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79749-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3173337026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-79612-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[39557]: pgmap v179: 420 pgs: 32 unknown, 388 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79749-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79749-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[39557]: osdmap e146: 8 total, 8 up, 8 in 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79749-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:06.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3173337026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-79612-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:06.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:05 smithi179 ceph-mon[33633]: pgmap v179: 420 pgs: 32 unknown, 388 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-14T17:49:06.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:06.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:05 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:06.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79749-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:06.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79749-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:06.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:05 smithi179 ceph-mon[33633]: osdmap e146: 8 total, 8 up, 8 in 2024-02-14T17:49:06.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79749-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:06.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3173337026' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-79612-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:07.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:07.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-79999-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]': finished 2024-02-14T17:49:07.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3173337026' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-79612-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[30138]: osdmap e147: 8 total, 8 up, 8 in 2024-02-14T17:49:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3162163143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi100-79590-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:07.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi100-79590-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-79999-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]': finished 2024-02-14T17:49:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3173337026' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-79612-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[39557]: osdmap e147: 8 total, 8 up, 8 in 2024-02-14T17:49:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3162163143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi100-79590-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:07.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi100-79590-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi100-79999-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]': finished 2024-02-14T17:49:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3173337026' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi100-79612-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:06 smithi179 ceph-mon[33633]: osdmap e147: 8 total, 8 up, 8 in 2024-02-14T17:49:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3162163143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi100-79590-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi100-79590-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:08.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:07 smithi100 ceph-mon[30138]: pgmap v182: 460 pgs: 40 unknown, 32 creating+peering, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:08.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:08.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79749-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79749-24"}]': finished 2024-02-14T17:49:08.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi100-79590-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:08.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:07 smithi100 ceph-mon[30138]: osdmap e148: 8 total, 8 up, 8 in 2024-02-14T17:49:08.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:07 smithi100 ceph-mon[39557]: pgmap v182: 460 pgs: 40 unknown, 32 creating+peering, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:08.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:08.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79749-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79749-24"}]': finished 2024-02-14T17:49:08.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi100-79590-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:08.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:07 smithi100 ceph-mon[39557]: osdmap e148: 8 total, 8 up, 8 in 2024-02-14T17:49:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:07 smithi179 ceph-mon[33633]: pgmap v182: 460 pgs: 40 unknown, 32 creating+peering, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79749-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79749-24"}]': finished 2024-02-14T17:49:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi100-79590-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:08.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:07 smithi179 ceph-mon[33633]: osdmap e148: 8 total, 8 up, 8 in 2024-02-14T17:49:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[30138]: osdmap e149: 8 total, 8 up, 8 in 2024-02-14T17:49:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1875557020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-79612-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-79612-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[39557]: osdmap e149: 8 total, 8 up, 8 in 2024-02-14T17:49:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1875557020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-79612-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-79612-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:08 smithi179 ceph-mon[33633]: osdmap e149: 8 total, 8 up, 8 in 2024-02-14T17:49:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1875557020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-79612-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-79612-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:09.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[39557]: pgmap v185: 468 pgs: 80 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-79612-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[39557]: osdmap e150: 8 total, 8 up, 8 in 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2577939380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-79590-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-79590-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[30138]: pgmap v185: 468 pgs: 80 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:10.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-79612-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:10.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:10.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[30138]: osdmap e150: 8 total, 8 up, 8 in 2024-02-14T17:49:10.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2577939380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-79590-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-79590-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:09 smithi179 ceph-mon[33633]: pgmap v185: 468 pgs: 80 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi100-79612-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:09 smithi179 ceph-mon[33633]: osdmap e150: 8 total, 8 up, 8 in 2024-02-14T17:49:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2577939380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-79590-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-79590-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:10.848 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:49:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:49:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:11.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:11.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:11.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-79590-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:11.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:10 smithi100 ceph-mon[39557]: osdmap e151: 8 total, 8 up, 8 in 2024-02-14T17:49:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:11.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:11.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-79590-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:11.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:10 smithi100 ceph-mon[30138]: osdmap e151: 8 total, 8 up, 8 in 2024-02-14T17:49:11.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 22 tests from LibRadosMisc api_snapshots_pp: Running main() from gmock_main.cc 2024-02-14T17:49:11.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: seed 79999 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (5967 ms) 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (5043 ms) 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (4125 ms) 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2095 ms) 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-14T17:49:11.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (4337 ms) 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (21567 ms total) 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (3828 ms) 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3863 ms) 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6067 ms) 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3946 ms) 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2062 ms) 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-14T17:49:11.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1430-g8727ce93/rpm/el9/BUILD/ceph-19.0.0-1430-g8727ce93/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi100-79999-7 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17952 ms) 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (37718 ms total) 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5057 ms) 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5065 ms total) 2024-02-14T17:49:11.285 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-02-14T17:49:11.286 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-14T17:49:11.286 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-14T17:49:11.286 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (2705 ms) 2024-02-14T17:49:11.286 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-14T17:49:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79749-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi100-79590-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:10 smithi179 ceph-mon[33633]: osdmap e151: 8 total, 8 up, 8 in 2024-02-14T17:49:11.851 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:49:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:49:11.851 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:11 smithi179 ceph-mon[33633]: pgmap v188: 468 pgs: 80 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:11.851 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:11.851 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:11.851 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:11 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:12.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:11 smithi179 ceph-mon[33633]: osdmap e152: 8 total, 8 up, 8 in 2024-02-14T17:49:12.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3441529018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79612-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:12.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79612-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:12.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[39557]: pgmap v188: 468 pgs: 80 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:12.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:12.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:12.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:12.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[39557]: osdmap e152: 8 total, 8 up, 8 in 2024-02-14T17:49:12.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3441529018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79612-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:12.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79612-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[30138]: pgmap v188: 468 pgs: 80 unknown, 388 active+clean; 592 KiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[30138]: osdmap e152: 8 total, 8 up, 8 in 2024-02-14T17:49:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3441529018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79612-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79612-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79612-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[39557]: osdmap e153: 8 total, 8 up, 8 in 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2729012669' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-79590-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-79590-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79612-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[30138]: osdmap e153: 8 total, 8 up, 8 in 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2729012669' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-79590-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-79590-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:13.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:12 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi100-79612-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:12 smithi179 ceph-mon[33633]: osdmap e153: 8 total, 8 up, 8 in 2024-02-14T17:49:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2729012669' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-79590-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-79590-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:14.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[30138]: pgmap v191: 460 pgs: 1 creating+peering, 63 unknown, 396 active+clean; 592 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:49:14.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:14.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-79590-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:14.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]': finished 2024-02-14T17:49:14.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[30138]: osdmap e154: 8 total, 8 up, 8 in 2024-02-14T17:49:14.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:14.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:14.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[39557]: pgmap v191: 460 pgs: 1 creating+peering, 63 unknown, 396 active+clean; 592 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:49:14.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:14.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-79590-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:14.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]': finished 2024-02-14T17:49:14.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[39557]: osdmap e154: 8 total, 8 up, 8 in 2024-02-14T17:49:14.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:14.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosS snapshots: Running main() from gmock_main.cc 2024-02-14T17:49:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-02-14T17:49:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [----------] Global test environment set-up. 2024-02-14T17:49:14.310 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (12813 ms) 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (8880 ms) 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (3840 ms) 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5422 ms) 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (6925 ms) 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (37880 ms total) 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-02-14T17:49:14.311 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (4994 ms) 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (7026 ms) 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7965 ms) 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5972 ms) 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4016 ms) 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi100-80886-11. 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: Waiting for snaps to purge. 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20025 ms) 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (49998 ms total) 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: 2024-02-14T17:49:14.312 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [----------] Global test environment tear-down 2024-02-14T17:49:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (87879 ms total) 2024-02-14T17:49:14.313 INFO:tasks.workunit.client.0.smithi100.stdout: snapshots: [ PASSED ] 11 tests. 2024-02-14T17:49:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:13 smithi179 ceph-mon[33633]: pgmap v191: 460 pgs: 1 creating+peering, 63 unknown, 396 active+clean; 592 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:49:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi100-79590-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79749-24"}]': finished 2024-02-14T17:49:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:13 smithi179 ceph-mon[33633]: osdmap e154: 8 total, 8 up, 8 in 2024-02-14T17:49:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2799158472' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]: dispatch 2024-02-14T17:49:15.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:15.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]': finished 2024-02-14T17:49:15.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:14 smithi100 ceph-mon[30138]: osdmap e155: 8 total, 8 up, 8 in 2024-02-14T17:49:15.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/475461496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-79612-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:15.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-79612-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:15.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:15.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]': finished 2024-02-14T17:49:15.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:14 smithi100 ceph-mon[39557]: osdmap e155: 8 total, 8 up, 8 in 2024-02-14T17:49:15.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/475461496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-79612-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:15.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-79612-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79749-24"}]': finished 2024-02-14T17:49:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:14 smithi179 ceph-mon[33633]: osdmap e155: 8 total, 8 up, 8 in 2024-02-14T17:49:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/475461496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-79612-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:15.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-79612-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[30138]: pgmap v194: 396 pgs: 32 unknown, 364 active+clean; 592 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:16.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-79612-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[30138]: osdmap e156: 8 total, 8 up, 8 in 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2307984660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-79590-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/521642702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-79749-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-79590-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-79749-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[39557]: pgmap v194: 396 pgs: 32 unknown, 364 active+clean; 592 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-79612-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[39557]: osdmap e156: 8 total, 8 up, 8 in 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2307984660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-79590-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/521642702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-79749-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-79590-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-79749-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:15 smithi179 ceph-mon[33633]: pgmap v194: 396 pgs: 32 unknown, 364 active+clean; 592 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:16.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:16.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi100-79612-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:16.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:15 smithi179 ceph-mon[33633]: osdmap e156: 8 total, 8 up, 8 in 2024-02-14T17:49:16.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2307984660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-79590-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/521642702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-79749-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-79590-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:16.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-79749-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:17.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:17.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-79590-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:17.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-79749-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:17.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:16 smithi100 ceph-mon[30138]: osdmap e157: 8 total, 8 up, 8 in 2024-02-14T17:49:17.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:17.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:17.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-79590-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:17.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-79749-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:17.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:16 smithi100 ceph-mon[39557]: osdmap e157: 8 total, 8 up, 8 in 2024-02-14T17:49:17.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi100-79590-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi100-79749-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:16 smithi179 ceph-mon[33633]: osdmap e157: 8 total, 8 up, 8 in 2024-02-14T17:49:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:18.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: pgmap v197: 420 pgs: 64 creating+peering, 356 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:18.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:18.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]': finished 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: osdmap e158: 8 total, 8 up, 8 in 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/225281177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-79612-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-79612-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: pgmap v197: 420 pgs: 64 creating+peering, 356 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:49:18.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:18.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]': finished 2024-02-14T17:49:18.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: osdmap e158: 8 total, 8 up, 8 in 2024-02-14T17:49:18.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:18.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/225281177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-79612-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:18.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-79612-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:18.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:49:18.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:49:18.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:17 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:49:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: pgmap v197: 420 pgs: 64 creating+peering, 356 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:49:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi100-79999-16"}]': finished 2024-02-14T17:49:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: osdmap e158: 8 total, 8 up, 8 in 2024-02-14T17:49:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-79999-16"}]: dispatch 2024-02-14T17:49:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/225281177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-79612-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-79612-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:49:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:49:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:17 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:49:19.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: pgmap v199: 388 pgs: 32 unknown, 356 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-79999-16"}]': finished 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-79612-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: osdmap e159: 8 total, 8 up, 8 in 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2460770685' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-79590-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2524056632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-79749-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:19.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: pgmap v199: 388 pgs: 32 unknown, 356 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-79999-16"}]': finished 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-79612-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: osdmap e159: 8 total, 8 up, 8 in 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2460770685' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-79590-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2524056632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-79749-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:19.153 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: pgmap v199: 388 pgs: 32 unknown, 356 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:49:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1160785251' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi100-79999-16"}]': finished 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi100-79612-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: osdmap e159: 8 total, 8 up, 8 in 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2460770685' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-79590-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2524056632' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-79749-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:20.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:20.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2460770685' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-79590-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2524056632' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-79749-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[39557]: osdmap e160: 8 total, 8 up, 8 in 2024-02-14T17:49:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:20.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2460770685' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-79590-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2524056632' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-79749-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:20.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[30138]: osdmap e160: 8 total, 8 up, 8 in 2024-02-14T17:49:20.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:20.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2460770685' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi100-79590-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2524056632' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi100-79749-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:19 smithi179 ceph-mon[33633]: osdmap e160: 8 total, 8 up, 8 in 2024-02-14T17:49:20.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:20.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:20.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:49:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:49:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[39557]: pgmap v202: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[39557]: osdmap e161: 8 total, 8 up, 8 in 2024-02-14T17:49:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2396932293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-79612-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-79612-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[30138]: pgmap v202: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[30138]: osdmap e161: 8 total, 8 up, 8 in 2024-02-14T17:49:21.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2396932293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-79612-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:21.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-79612-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:21.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:21 smithi179 ceph-mon[33633]: pgmap v202: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:21 smithi179 ceph-mon[33633]: osdmap e161: 8 total, 8 up, 8 in 2024-02-14T17:49:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2396932293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-79612-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-79612-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:22.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:49:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:49:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]': finished 2024-02-14T17:49:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-79612-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[30138]: osdmap e162: 8 total, 8 up, 8 in 2024-02-14T17:49:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3465743131' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-79590-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/161029190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-79749-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-79749-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[30138]: pgmap v205: 460 pgs: 72 unknown, 32 creating+peering, 356 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]': finished 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-79612-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[39557]: osdmap e162: 8 total, 8 up, 8 in 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3465743131' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-79590-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/161029190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-79749-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-79749-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[39557]: pgmap v205: 460 pgs: 72 unknown, 32 creating+peering, 356 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:22.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]': finished 2024-02-14T17:49:22.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi100-79612-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:22 smithi179 ceph-mon[33633]: osdmap e162: 8 total, 8 up, 8 in 2024-02-14T17:49:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3465743131' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-79590-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/161029190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-79749-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-79749-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:22 smithi179 ceph-mon[33633]: pgmap v205: 460 pgs: 72 unknown, 32 creating+peering, 356 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:22 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:23 smithi100 ceph-mon[30138]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3465743131' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-79590-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:23 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-79749-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:23 smithi100 ceph-mon[30138]: osdmap e163: 8 total, 8 up, 8 in 2024-02-14T17:49:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:23.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:23 smithi100 ceph-mon[39557]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:23.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3465743131' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-79590-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:23.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:23 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-79749-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:23.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:23 smithi100 ceph-mon[39557]: osdmap e163: 8 total, 8 up, 8 in 2024-02-14T17:49:23.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:23 smithi179 ceph-mon[33633]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3465743131' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi100-79590-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:23 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi100-79749-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:23 smithi179 ceph-mon[33633]: osdmap e163: 8 total, 8 up, 8 in 2024-02-14T17:49:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: osdmap e164: 8 total, 8 up, 8 in 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/119092731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-79612-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-79612-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-79590-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: pgmap v208: 396 pgs: 40 unknown, 356 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: osdmap e164: 8 total, 8 up, 8 in 2024-02-14T17:49:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/119092731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-79612-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-79612-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-79590-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: pgmap v208: 396 pgs: 40 unknown, 356 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: osdmap e164: 8 total, 8 up, 8 in 2024-02-14T17:49:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/119092731' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-79612-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-79612-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-79590-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: pgmap v208: 396 pgs: 40 unknown, 356 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:26.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-79612-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:26.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-79590-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[39557]: osdmap e165: 8 total, 8 up, 8 in 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-79590-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-79749-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-79612-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-79590-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:26.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[30138]: osdmap e165: 8 total, 8 up, 8 in 2024-02-14T17:49:26.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-79590-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:26.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-79749-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:26.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:26.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:26.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi100-79612-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi100-79590-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:25 smithi179 ceph-mon[33633]: osdmap e165: 8 total, 8 up, 8 in 2024-02-14T17:49:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-79590-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-79749-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:26.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: pgmap v211: 364 pgs: 8 unknown, 356 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: osdmap e166: 8 total, 8 up, 8 in 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-79590-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-79590-27"}]': finished 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-79749-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-79749-36"}]': finished 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13"}]': finished 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: osdmap e167: 8 total, 8 up, 8 in 2024-02-14T17:49:27.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1304132119' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-79612-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: pgmap v211: 364 pgs: 8 unknown, 356 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: osdmap e166: 8 total, 8 up, 8 in 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-79590-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-79590-27"}]': finished 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-79749-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-79749-36"}]': finished 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13"}]': finished 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: osdmap e167: 8 total, 8 up, 8 in 2024-02-14T17:49:27.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1304132119' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-79612-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: pgmap v211: 364 pgs: 8 unknown, 356 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: osdmap e166: 8 total, 8 up, 8 in 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13"}]: dispatch 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi100-79590-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi100-79590-27"}]': finished 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi100-79749-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi100-79749-36"}]': finished 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-13"}]': finished 2024-02-14T17:49:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: osdmap e167: 8 total, 8 up, 8 in 2024-02-14T17:49:27.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1304132119' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-79612-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:28.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1304132119' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-79612-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:28.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:27 smithi100 ceph-mon[30138]: osdmap e168: 8 total, 8 up, 8 in 2024-02-14T17:49:28.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:28.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1304132119' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-79612-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:28.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:27 smithi100 ceph-mon[39557]: osdmap e168: 8 total, 8 up, 8 in 2024-02-14T17:49:28.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:28.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1304132119' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi100-79612-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:28.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:27 smithi179 ceph-mon[33633]: osdmap e168: 8 total, 8 up, 8 in 2024-02-14T17:49:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[30138]: pgmap v214: 380 pgs: 48 unknown, 332 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-02-14T17:49:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[30138]: osdmap e169: 8 total, 8 up, 8 in 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[39557]: pgmap v214: 380 pgs: 48 unknown, 332 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[39557]: osdmap e169: 8 total, 8 up, 8 in 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:29.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:29.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:28 smithi179 ceph-mon[33633]: pgmap v214: 380 pgs: 48 unknown, 332 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-02-14T17:49:29.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:29.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:28 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:29.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:28 smithi179 ceph-mon[33633]: osdmap e169: 8 total, 8 up, 8 in 2024-02-14T17:49:29.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:29.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:29.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:29.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:30.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:30.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36"}]': finished 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-79590-27"}]': finished 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[39557]: osdmap e170: 8 total, 8 up, 8 in 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3382586733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-79612-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36"}]': finished 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-79590-27"}]': finished 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[30138]: osdmap e170: 8 total, 8 up, 8 in 2024-02-14T17:49:30.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:30.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:30.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3382586733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-79612-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:30.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:30.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi100-79749-36"}]': finished 2024-02-14T17:49:30.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi100-79590-27"}]': finished 2024-02-14T17:49:30.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:30.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:29 smithi179 ceph-mon[33633]: osdmap e170: 8 total, 8 up, 8 in 2024-02-14T17:49:30.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-79590-27"}]: dispatch 2024-02-14T17:49:30.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-79749-36"}]: dispatch 2024-02-14T17:49:30.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3382586733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-79612-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: Running main() from gmock_main.cc 2024-02-14T17:49:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-02-14T17:49:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] Global test environment set-up. 2024-02-14T17:49:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-02-14T17:49:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-14T17:49:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.TooBig (10440 ms) 2024-02-14T17:49:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-14T17:49:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (7401 ms) 2024-02-14T17:49:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-14T17:49:30.384 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (5041 ms) 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (5189 ms) 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (5211 ms) 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3020 ms) 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3035 ms) 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2929 ms) 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3046 ms) 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (4032 ms) 2024-02-14T17:49:30.385 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2928 ms) 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3009 ms) 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3042 ms) 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3004 ms) 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.Flush (2947 ms) 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3013 ms) 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3013 ms) 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-14T17:49:30.386 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3011 ms) 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3020 ms) 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3254 ms) 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (2697 ms) 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3011 ms) 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3016 ms) 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3023 ms) 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3258 ms) 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-14T17:49:30.387 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (2781 ms) 2024-02-14T17:49:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] 26 tests from LibRadosAio (97372 ms total) 2024-02-14T17:49:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: 2024-02-14T17:49:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-14T17:49:30.388 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout:PP (68507 ms total) 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (121 ms) 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (121 ms total) 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (41 ms) 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-14T17:49:30.394 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (12 ms) 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (53 ms total) 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (46 ms) 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (7 ms) 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (53 ms total) 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (60 ms) 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (16 ms) 2024-02-14T17:49:30.395 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (76 ms total) 2024-02-14T17:49:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-02-14T17:49:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-14T17:49:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-14T17:49:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1044 ms) 2024-02-14T17:49:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1044 ms total) 2024-02-14T17:49:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: 2024-02-14T17:49:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-14T17:49:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (104320 ms total) 2024-02-14T17:49:30.396 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-14T17:49:30.844 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:49:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:31.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: pgmap v217: 396 pgs: 64 unknown, 332 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-02-14T17:49:31.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:31.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-79590-27"}]': finished 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-79749-36"}]': finished 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3382586733' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-79612-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: osdmap e171: 8 total, 8 up, 8 in 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-79590-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: pgmap v217: 396 pgs: 64 unknown, 332 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-79590-27"}]': finished 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-79749-36"}]': finished 2024-02-14T17:49:31.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3382586733' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-79612-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:31.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: osdmap e171: 8 total, 8 up, 8 in 2024-02-14T17:49:31.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:31.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:31.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:31.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:31.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-79590-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: pgmap v217: 396 pgs: 64 unknown, 332 active+clean; 585 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-02-14T17:49:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1239908079' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi100-79590-27"}]': finished 2024-02-14T17:49:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3546877243' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi100-79749-36"}]': finished 2024-02-14T17:49:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3382586733' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi100-79612-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: osdmap e171: 8 total, 8 up, 8 in 2024-02-14T17:49:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-79590-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:32.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:32.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-79590-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:31 smithi179 ceph-mon[33633]: osdmap e172: 8 total, 8 up, 8 in 2024-02-14T17:49:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-79590-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:32.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:49:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:49:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-79590-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[30138]: osdmap e172: 8 total, 8 up, 8 in 2024-02-14T17:49:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-79590-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:32.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:32.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:32.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi100-79590-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:32.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[39557]: osdmap e172: 8 total, 8 up, 8 in 2024-02-14T17:49:32.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-79590-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:32.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:32.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[30138]: pgmap v220: 364 pgs: 364 active+clean; 617 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:49:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-15"}]': finished 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-15", "mode": "writeback"}]: dispatch 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[30138]: osdmap e173: 8 total, 8 up, 8 in 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-15", "mode": "writeback"}]: dispatch 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1452654126' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-79612-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-79612-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[39557]: pgmap v220: 364 pgs: 364 active+clean; 617 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-15"}]': finished 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-15", "mode": "writeback"}]: dispatch 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[39557]: osdmap e173: 8 total, 8 up, 8 in 2024-02-14T17:49:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-15", "mode": "writeback"}]: dispatch 2024-02-14T17:49:33.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1452654126' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-79612-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:33.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-79612-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:32 smithi179 ceph-mon[33633]: pgmap v220: 364 pgs: 364 active+clean; 617 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:49:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:32 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-15"}]': finished 2024-02-14T17:49:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-15", "mode": "writeback"}]: dispatch 2024-02-14T17:49:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:32 smithi179 ceph-mon[33633]: osdmap e173: 8 total, 8 up, 8 in 2024-02-14T17:49:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-15", "mode": "writeback"}]: dispatch 2024-02-14T17:49:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1452654126' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-79612-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:33.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-79612-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-79590-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-79590-28"}]': finished 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-15", "mode": "writeback"}]': finished 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-79612-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: osdmap e174: 8 total, 8 up, 8 in 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:34.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-79590-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-79590-28"}]': finished 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-15", "mode": "writeback"}]': finished 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-79612-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: osdmap e174: 8 total, 8 up, 8 in 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:34.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:34.280 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-02-14T17:49:34.280 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-02-14T17:49:34.280 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-14T17:49:34.280 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (10145 ms) 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (19259 ms) 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6904 ms) 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3026 ms) 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (2935 ms) 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3047 ms) 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3014 ms) 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-14T17:49:34.281 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3009 ms) 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2936 ms) 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3019 ms) 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3023 ms) 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2951 ms) 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3012 ms) 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3015 ms) 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3010 ms) 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3014 ms) 2024-02-14T17:49:34.282 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3016 ms) 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3018 ms) 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (2935 ms) 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3020 ms) 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3010 ms) 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3018 ms) 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3019 ms) 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-14T17:49:34.283 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3486 ms) 2024-02-14T17:49:34.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-14T17:49:34.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2541 ms) 2024-02-14T17:49:34.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-14T17:49:34.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3009 ms) 2024-02-14T17:49:34.284 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-14T17:49:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi100-79590-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi100-79590-28"}]': finished 2024-02-14T17:49:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-15", "mode": "writeback"}]': finished 2024-02-14T17:49:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi100-79612-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:49:34.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: osdmap e174: 8 total, 8 up, 8 in 2024-02-14T17:49:34.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:34.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:34.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:34.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:35.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[30138]: pgmap v223: 404 pgs: 40 unknown, 364 active+clean; 617 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[30138]: osdmap e175: 8 total, 8 up, 8 in 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[39557]: pgmap v223: 404 pgs: 40 unknown, 364 active+clean; 617 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:49:35.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[39557]: osdmap e175: 8 total, 8 up, 8 in 2024-02-14T17:49:35.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:35.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:35.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:35.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:35.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:34 smithi179 ceph-mon[33633]: pgmap v223: 404 pgs: 40 unknown, 364 active+clean; 617 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-02-14T17:49:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:34 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:49:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:34 smithi179 ceph-mon[33633]: osdmap e175: 8 total, 8 up, 8 in 2024-02-14T17:49:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:36.295 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAinapshotsECPP.SnapRemovePP (1999 ms) 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2016 ms) 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2004 ms) 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8724 ms total) 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4499 ms) 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3547 ms) 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3890 ms) 2024-02-14T17:49:36.296 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11936 ms total) 2024-02-14T17:49:36.297 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: 2024-02-14T17:49:36.297 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-14T17:49:36.297 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (110164 ms total) 2024-02-14T17:49:36.297 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-14T17:49:36.297 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-02-14T17:49:36.297 INFO:tasks.workunit.client.0.smithi100.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-14T17:49:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]': finished 2024-02-14T17:49:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: osdmap e176: 8 total, 8 up, 8 in 2024-02-14T17:49:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3840185410' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-79612-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-79612-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:36 smithi179 ceph-mon[33633]: pgmap v226: 388 pgs: 32 unknown, 356 active+clean; 585 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]': finished 2024-02-14T17:49:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: osdmap e176: 8 total, 8 up, 8 in 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3840185410' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-79612-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-79612-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[30138]: pgmap v226: 388 pgs: 32 unknown, 356 active+clean; 585 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]': finished 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: osdmap e176: 8 total, 8 up, 8 in 2024-02-14T17:49:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3814404115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]: dispatch 2024-02-14T17:49:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3840185410' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-79612-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-79612-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15"}]: dispatch 2024-02-14T17:49:36.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:36 smithi100 ceph-mon[39557]: pgmap v226: 388 pgs: 32 unknown, 356 active+clean; 585 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:37 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-79590-28"}]': finished 2024-02-14T17:49:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]': finished 2024-02-14T17:49:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-79612-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15"}]': finished 2024-02-14T17:49:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:37 smithi179 ceph-mon[33633]: osdmap e177: 8 total, 8 up, 8 in 2024-02-14T17:49:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-79590-28"}]': finished 2024-02-14T17:49:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]': finished 2024-02-14T17:49:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-79612-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15"}]': finished 2024-02-14T17:49:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[30138]: osdmap e177: 8 total, 8 up, 8 in 2024-02-14T17:49:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi100-79590-28"}]': finished 2024-02-14T17:49:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi100-79999-21"}]': finished 2024-02-14T17:49:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi100-79612-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-15"}]': finished 2024-02-14T17:49:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[39557]: osdmap e177: 8 total, 8 up, 8 in 2024-02-14T17:49:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-79590-28"}]: dispatch 2024-02-14T17:49:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-79590-28"}]': finished 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: osdmap e178: 8 total, 8 up, 8 in 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: pgmap v229: 324 pgs: 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-79590-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-79590-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-79590-28"}]': finished 2024-02-14T17:49:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: osdmap e178: 8 total, 8 up, 8 in 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: pgmap v229: 324 pgs: 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-79590-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-79590-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4101422555' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi100-79590-28"}]': finished 2024-02-14T17:49:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: osdmap e178: 8 total, 8 up, 8 in 2024-02-14T17:49:38.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: pgmap v229: 324 pgs: 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:38.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:38.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:38.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:38.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-79590-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:38.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-79590-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:39.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-14T17:49:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-79590-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: osdmap e179: 8 total, 8 up, 8 in 2024-02-14T17:49:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-79590-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-79590-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/941062206' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-79612-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-79612-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:39.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:39.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-14T17:49:39.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-79590-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:39.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: osdmap e179: 8 total, 8 up, 8 in 2024-02-14T17:49:39.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-79590-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:39.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-79590-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:39.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/941062206' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-79612-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-79612-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:39.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:39.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-14T17:49:39.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi100-79590-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:39.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: osdmap e179: 8 total, 8 up, 8 in 2024-02-14T17:49:39.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-79590-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:39.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-79590-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:39.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/941062206' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-79612-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-79612-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:39.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:40.803 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:49:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-79612-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: osdmap e180: 8 total, 8 up, 8 in 2024-02-14T17:49:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: pgmap v232: 388 pgs: 64 unknown, 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-79590-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-79590-29"}]': finished 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: osdmap e181: 8 total, 8 up, 8 in 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:40.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-79612-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: osdmap e180: 8 total, 8 up, 8 in 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: pgmap v232: 388 pgs: 64 unknown, 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-79590-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-79590-29"}]': finished 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: osdmap e181: 8 total, 8 up, 8 in 2024-02-14T17:49:40.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:40.806 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:40.806 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:40.806 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:40.806 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:40.806 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi100-79612-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: osdmap e180: 8 total, 8 up, 8 in 2024-02-14T17:49:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]: dispatch 2024-02-14T17:49:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: pgmap v232: 388 pgs: 64 unknown, 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi100-79590-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi100-79590-29"}]': finished 2024-02-14T17:49:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pg_num","val":"11"}]': finished 2024-02-14T17:49:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: osdmap e181: 8 total, 8 up, 8 in 2024-02-14T17:49:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:41.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-14T17:49:42.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:49:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:49:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:49:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-17"}]': finished 2024-02-14T17:49:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:49:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-17", "mode": "writeback"}]: dispatch 2024-02-14T17:49:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: osdmap e182: 8 total, 8 up, 8 in 2024-02-14T17:49:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2017013431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-79612-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-17", "mode": "writeback"}]: dispatch 2024-02-14T17:49:42.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-79612-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:42.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: pgmap v235: 396 pgs: 32 unknown, 8 creating+peering, 356 active+clean; 585 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-14T17:49:42.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:42.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:42.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:42 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:42.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:49:42.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-17"}]': finished 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-17", "mode": "writeback"}]: dispatch 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: osdmap e182: 8 total, 8 up, 8 in 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2017013431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-79612-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-17", "mode": "writeback"}]: dispatch 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-79612-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: pgmap v235: 396 pgs: 32 unknown, 8 creating+peering, 356 active+clean; 585 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-17"}]': finished 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi100-79670-1","var":"pgp_num","val":"11"}]': finished 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-17", "mode": "writeback"}]: dispatch 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: osdmap e182: 8 total, 8 up, 8 in 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2017013431' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-79612-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-17", "mode": "writeback"}]: dispatch 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-79612-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: pgmap v235: 396 pgs: 32 unknown, 8 creating+peering, 356 active+clean; 585 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-14T17:49:42.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:42 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-17", "mode": "writeback"}]': finished 2024-02-14T17:49:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-79612-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-14T17:49:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: osdmap e183: 8 total, 8 up, 8 in 2024-02-14T17:49:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-17", "mode": "writeback"}]': finished 2024-02-14T17:49:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-79612-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: osdmap e183: 8 total, 8 up, 8 in 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-17", "mode": "writeback"}]': finished 2024-02-14T17:49:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi100-79612-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-14T17:49:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: osdmap e183: 8 total, 8 up, 8 in 2024-02-14T17:49:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:43.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]': finished 2024-02-14T17:49:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: osdmap e184: 8 total, 8 up, 8 in 2024-02-14T17:49:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: pgmap v238: 356 pgs: 356 active+clean; 585 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-14T17:49:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-14T17:49:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:44 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-14T17:49:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]': finished 2024-02-14T17:49:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: osdmap e184: 8 total, 8 up, 8 in 2024-02-14T17:49:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: pgmap v238: 356 pgs: 356 active+clean; 585 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi100-79590-29"}]': finished 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: osdmap e184: 8 total, 8 up, 8 in 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:44.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075415193' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:44.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17"}]: dispatch 2024-02-14T17:49:44.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]: dispatch 2024-02-14T17:49:44.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:44.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: pgmap v238: 356 pgs: 356 active+clean; 585 KiB data, 575 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-14T17:49:44.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-14T17:49:44.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:44 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-14T17:49:45.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:45.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17"}]': finished 2024-02-14T17:49:45.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]': finished 2024-02-14T17:49:45.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-14T17:49:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: osdmap e185: 8 total, 8 up, 8 in 2024-02-14T17:49:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/549839054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-79612-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-79612-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-79590-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-79590-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:45.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:45.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17"}]': finished 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]': finished 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: osdmap e185: 8 total, 8 up, 8 in 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/549839054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-79612-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-79612-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-79590-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-79590-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:45.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-17"}]': finished 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi100-79590-29"}]': finished 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi100-79670-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: osdmap e185: 8 total, 8 up, 8 in 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/549839054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-79612-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-79612-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-79590-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-79590-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:45.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3795827643' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-79612-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-79590-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: osdmap e186: 8 total, 8 up, 8 in 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-79590-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-79590-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: pgmap v241: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/549839054' entity='client.admin' cmd=[{ 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "prefix": "osd pool set", 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "pool": "PoolEIOFlag_smithi100-79612-33", 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "var": "eio", 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "val": "true" 2024-02-14T17:49:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: }]: dispatch 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "prefix": "osd pool set", 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "pool": "PoolEIOFlag_smithi100-79612-33", 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "var": "eio", 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "val": "true" 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: }]: dispatch 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "prefix": "osd pool set", 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "pool": "PoolEIOFlag_smithi100-79612-33", 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "var": "eio", 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: "val": "true" 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: }]': finished 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: osdmap e187: 8 total, 8 up, 8 in 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:46.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:46.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-79612-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:46.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-79590-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: osdmap e186: 8 total, 8 up, 8 in 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-79590-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-79590-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: pgmap v241: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/549839054' entity='client.admin' cmd=[{ 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "prefix": "osd pool set", 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "pool": "PoolEIOFlag_smithi100-79612-33", 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "var": "eio", 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "val": "true" 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: }]: dispatch 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "prefix": "osd pool set", 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "pool": "PoolEIOFlag_smithi100-79612-33", 2024-02-14T17:49:46.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "var": "eio", 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "val": "true" 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: }]: dispatch 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "prefix": "osd pool set", 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "pool": "PoolEIOFlag_smithi100-79612-33", 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "var": "eio", 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: "val": "true" 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: }]': finished 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: osdmap e187: 8 total, 8 up, 8 in 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi100-79612-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:46.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi100-79590-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: osdmap e186: 8 total, 8 up, 8 in 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-79590-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-79590-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: pgmap v241: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 575 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/549839054' entity='client.admin' cmd=[{ 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "prefix": "osd pool set", 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "pool": "PoolEIOFlag_smithi100-79612-33", 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "var": "eio", 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "val": "true" 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: }]: dispatch 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "prefix": "osd pool set", 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "pool": "PoolEIOFlag_smithi100-79612-33", 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "var": "eio", 2024-02-14T17:49:46.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "val": "true" 2024-02-14T17:49:46.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: }]: dispatch 2024-02-14T17:49:46.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-14T17:49:46.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "prefix": "osd pool set", 2024-02-14T17:49:46.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "pool": "PoolEIOFlag_smithi100-79612-33", 2024-02-14T17:49:46.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "var": "eio", 2024-02-14T17:49:46.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: "val": "true" 2024-02-14T17:49:46.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: }]': finished 2024-02-14T17:49:46.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: osdmap e187: 8 total, 8 up, 8 in 2024-02-14T17:49:46.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:46.904 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:47.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-79670-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-79670-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-79590-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-79590-30"}]': finished 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-79670-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: osdmap e188: 8 total, 8 up, 8 in 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-79670-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-79670-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-79670-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-79670-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-79590-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-79590-30"}]': finished 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-79670-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: osdmap e188: 8 total, 8 up, 8 in 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-79670-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-79670-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-79670-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-79670-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi100-79590-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi100-79590-30"}]': finished 2024-02-14T17:49:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi100-79670-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: osdmap e188: 8 total, 8 up, 8 in 2024-02-14T17:49:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-79670-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-79670-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:48 smithi179 ceph-mon[33633]: pgmap v244: 332 pgs: 1 creating+peering, 39 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:48 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:48 smithi179 ceph-mon[33633]: osdmap e189: 8 total, 8 up, 8 in 2024-02-14T17:49:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4062208024' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-79612-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-79612-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[30138]: pgmap v244: 332 pgs: 1 creating+peering, 39 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[30138]: osdmap e189: 8 total, 8 up, 8 in 2024-02-14T17:49:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4062208024' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-79612-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:48.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-79612-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[39557]: pgmap v244: 332 pgs: 1 creating+peering, 39 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[39557]: osdmap e189: 8 total, 8 up, 8 in 2024-02-14T17:49:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4062208024' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-79612-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-79612-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:50.373 INFO:tasks.workunit.client.0.smithi100.stdout:ry=12 expected=12 2024-02-14T17:49:50.373 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:d83876eb:::4:head 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:d83876eb:::4:head expected=38:d83876eb:::4:head 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:d83876eb:::4:head -> 4 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=4 expected=4 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:cfc208b3:::3:head 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:cfc208b3:::3:head expected=38:cfc208b3:::3:head 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:cfc208b3:::3:head -> 3 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=3 expected=3 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:c4fdafeb:::6:head 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:c4fdafeb:::6:head expected=38:c4fdafeb:::6:head 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:c4fdafeb:::6:head -> 6 2024-02-14T17:49:50.374 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=6 expected=6 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:62a1935d:::14:head 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:62a1935d:::14:head expected=38:62a1935d:::14:head 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:62a1935d:::14:head -> 14 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=14 expected=14 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:52ea6a34:::10:head 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:52ea6a34:::10:head expected=38:52ea6a34:::10:head 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:52ea6a34:::10:head -> 10 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=10 expected=10 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:5c6b0b28:::7:head 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:5c6b0b28:::7:head expected=38:5c6b0b28:::7:head 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:5c6b0b28:::7:head -> 7 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=7 expected=7 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:566253c9:::13:head 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:566253c9:::13:head expected=38:566253c9:::13:head 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:566253c9:::13:head -> 13 2024-02-14T17:49:50.375 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=13 expected=13 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : seek to 38:02547ec2:::1:head 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : cursor()=38:02547ec2:::1:head expected=38:02547ec2:::1:head 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: > 38:02547ec2:::1:head -> 1 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: : entry=1 expected=1 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (410 ms) 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (102628 ms) 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: split 0/5 -> MIN 38:33333333::::head 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: split 1/5 -> 38:33333333::::head 38:66666666::::head 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: split 2/5 -> 38:66666666::::head 38:99999999::::head 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: split 3/5 -> 38:99999999::::head 38:cccccccc::::head 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: split 4/5 -> 38:cccccccc::::head MAX 2024-02-14T17:49:50.376 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (5961 ms) 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 7 tests from LibRadosList (109527 ms total) 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1046 ms) 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo1,foo2,foo3 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo1 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo2 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo3 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo1,foo4,foo5 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo4 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo5 2024-02-14T17:49:50.377 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo1 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset foo6,foo7 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo7 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: foo6 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo4 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo5 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns2:foo7 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns2:foo6 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: ns1:foo1 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo1 2024-02-14T17:49:50.378 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo2 2024-02-14T17:49:50.379 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: :foo3 2024-02-14T17:49:50.379 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (35 ms) 2024-02-14T17:49:50.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-79670-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-79670-2"}]': finished 2024-02-14T17:49:50.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-19"}]': finished 2024-02-14T17:49:50.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-79612-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:50 smithi179 ceph-mon[33633]: osdmap e190: 8 total, 8 up, 8 in 2024-02-14T17:49:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-19", "mode": "writeback"}]: dispatch 2024-02-14T17:49:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-19", "mode": "writeback"}]: dispatch 2024-02-14T17:49:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:50 smithi179 ceph-mon[33633]: pgmap v247: 364 pgs: 1 creating+peering, 71 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:50.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:49:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:49:50.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-79670-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-79670-2"}]': finished 2024-02-14T17:49:50.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-19"}]': finished 2024-02-14T17:49:50.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-79612-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:50.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[30138]: osdmap e190: 8 total, 8 up, 8 in 2024-02-14T17:49:50.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-19", "mode": "writeback"}]: dispatch 2024-02-14T17:49:50.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-19", "mode": "writeback"}]: dispatch 2024-02-14T17:49:50.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:50.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:50.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[30138]: pgmap v247: 364 pgs: 1 creating+peering, 71 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:50.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi100-79670-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi100-79670-2"}]': finished 2024-02-14T17:49:50.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-19"}]': finished 2024-02-14T17:49:50.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi100-79612-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:50.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[39557]: osdmap e190: 8 total, 8 up, 8 in 2024-02-14T17:49:50.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-19", "mode": "writeback"}]: dispatch 2024-02-14T17:49:50.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-19", "mode": "writeback"}]: dispatch 2024-02-14T17:49:50.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:50.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:50.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:50 smithi100 ceph-mon[39557]: pgmap v247: 364 pgs: 1 creating+peering, 71 unknown, 292 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:49:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:51 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-19", "mode": "writeback"}]': finished 2024-02-14T17:49:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]': finished 2024-02-14T17:49:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:51 smithi179 ceph-mon[33633]: osdmap e191: 8 total, 8 up, 8 in 2024-02-14T17:49:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:51.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:51.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-19", "mode": "writeback"}]': finished 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]': finished 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[30138]: osdmap e191: 8 total, 8 up, 8 in 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-19", "mode": "writeback"}]': finished 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi100-79590-30"}]': finished 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[39557]: osdmap e191: 8 total, 8 up, 8 in 2024-02-14T17:49:51.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2576693133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:51.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]: dispatch 2024-02-14T17:49:51.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:51.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:52.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:49:51 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:49:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]': finished 2024-02-14T17:49:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: osdmap e192: 8 total, 8 up, 8 in 2024-02-14T17:49:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/689171275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-79612-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-79612-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-79590-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-79590-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: pgmap v250: 356 pgs: 32 unknown, 324 active+clean; 583 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-79612-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19"}]': finished 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]': finished 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-79590-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:52.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:52 smithi179 ceph-mon[33633]: osdmap e193: 8 total, 8 up, 8 in 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]': finished 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: osdmap e192: 8 total, 8 up, 8 in 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/689171275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-79612-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-79612-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-79590-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:52.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-79590-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:52.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: pgmap v250: 356 pgs: 32 unknown, 324 active+clean; 583 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:49:52.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:52.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:52.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-79612-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:52.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19"}]': finished 2024-02-14T17:49:52.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi100-79590-30"}]': finished 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: osdmap e192: 8 total, 8 up, 8 in 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/689171275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-79612-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-79612-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19"}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-79590-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:52.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-79590-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:52.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: pgmap v250: 356 pgs: 32 unknown, 324 active+clean; 583 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:49:52.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:49:52.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:49:52.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi100-79612-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:52.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-19"}]': finished 2024-02-14T17:49:52.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]': finished 2024-02-14T17:49:52.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-79590-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:52.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[39557]: osdmap e193: 8 total, 8 up, 8 in 2024-02-14T17:49:52.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi100-79670-2"}]': finished 2024-02-14T17:49:52.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi100-79590-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:49:52.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:52 smithi100 ceph-mon[30138]: osdmap e193: 8 total, 8 up, 8 in 2024-02-14T17:49:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-79590-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:54.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:54.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-79590-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:54.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:54 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:54.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]': finished 2024-02-14T17:49:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-79590-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-79590-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]': finished 2024-02-14T17:49:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1802163757' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-79590-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]: dispatch 2024-02-14T17:49:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-79590-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi100-79670-2"}]': finished 2024-02-14T17:49:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:55 smithi179 ceph-mon[33633]: osdmap e194: 8 total, 8 up, 8 in 2024-02-14T17:49:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:55 smithi179 ceph-mon[33633]: pgmap v253: 292 pgs: 292 active+clean; 583 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-79590-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-79590-31"}]': finished 2024-02-14T17:49:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:55 smithi179 ceph-mon[33633]: osdmap e195: 8 total, 8 up, 8 in 2024-02-14T17:49:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3327019080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1309013272' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79670-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79670-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[39557]: osdmap e194: 8 total, 8 up, 8 in 2024-02-14T17:49:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[39557]: pgmap v253: 292 pgs: 292 active+clean; 583 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-79590-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-79590-31"}]': finished 2024-02-14T17:49:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[39557]: osdmap e195: 8 total, 8 up, 8 in 2024-02-14T17:49:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3327019080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1309013272' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79670-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79670-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[30138]: osdmap e194: 8 total, 8 up, 8 in 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[30138]: pgmap v253: 292 pgs: 292 active+clean; 583 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi100-79590-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi100-79590-31"}]': finished 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[30138]: osdmap e195: 8 total, 8 up, 8 in 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3327019080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1309013272' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79670-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:55.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79670-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:56.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-02-14T17:49:56.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 1 0 2024-02-14T17:49:56.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 10 0 2024-02-14T17:49:56.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 13 0 2024-02-14T17:49:56.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 7 0 2024-02-14T17:49:56.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 14 0 2024-02-14T17:49:56.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 0 0 2024-02-14T17:49:56.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 15 0 2024-02-14T17:49:56.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 11 0 2024-02-14T17:49:56.331 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 5 0 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 8 0 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 6 0 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 3 0 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 4 0 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 12 0 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 9 0 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 2 0 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (45 ms) 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 3 tests from LibRadosListEC (1126 ms total) 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-14T17:49:56.332 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-14T17:49:56.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3019 ms) 2024-02-14T17:49:56.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] 1 test from LibRadosListNP (3019 ms total) 2024-02-14T17:49:56.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: 2024-02-14T17:49:56.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [----------] Global test environment tear-down 2024-02-14T17:49:56.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [==========] 11 tests from 3 test suites ran. (130299 ms total) 2024-02-14T17:49:56.333 INFO:tasks.workunit.client.0.smithi100.stdout: api_list: [ PASSED ] 11 tests. 2024-02-14T17:49:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79670-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:56 smithi179 ceph-mon[33633]: osdmap e196: 8 total, 8 up, 8 in 2024-02-14T17:49:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1309013272' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-79670-3","pool2":"test-rados-api-smithi100-79670-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:49:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-79670-3","pool2":"test-rados-api-smithi100-79670-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:49:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:56 smithi179 ceph-mon[33633]: pgmap v256: 396 pgs: 104 unknown, 292 active+clean; 583 KiB data, 568 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:56 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79670-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[39557]: osdmap e196: 8 total, 8 up, 8 in 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1309013272' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-79670-3","pool2":"test-rados-api-smithi100-79670-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-79670-3","pool2":"test-rados-api-smithi100-79670-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[39557]: pgmap v256: 396 pgs: 104 unknown, 292 active+clean; 583 KiB data, 568 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79670-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[30138]: osdmap e196: 8 total, 8 up, 8 in 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:56.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:49:56.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1309013272' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-79670-3","pool2":"test-rados-api-smithi100-79670-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:49:56.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-79670-3","pool2":"test-rados-api-smithi100-79670-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-14T17:49:56.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[30138]: pgmap v256: 396 pgs: 104 unknown, 292 active+clean; 583 KiB data, 568 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:49:56.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:56 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:49:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-79670-3","pool2":"test-rados-api-smithi100-79670-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-14T17:49:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[30138]: osdmap e197: 8 total, 8 up, 8 in 2024-02-14T17:49:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/413901247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-79670-3","pool2":"test-rados-api-smithi100-79670-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[39557]: osdmap e197: 8 total, 8 up, 8 in 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/413901247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:49:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:49:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi100-79670-3","pool2":"test-rados-api-smithi100-79670-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-14T17:49:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:57 smithi179 ceph-mon[33633]: osdmap e197: 8 total, 8 up, 8 in 2024-02-14T17:49:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/413901247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:49:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:49:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]': finished 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-21"}]': finished 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[30138]: osdmap e198: 8 total, 8 up, 8 in 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-21", "mode": "writeback"}]: dispatch 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-21", "mode": "writeback"}]: dispatch 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[30138]: pgmap v259: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]': finished 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:58.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-21"}]': finished 2024-02-14T17:49:58.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[39557]: osdmap e198: 8 total, 8 up, 8 in 2024-02-14T17:49:58.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-21", "mode": "writeback"}]: dispatch 2024-02-14T17:49:58.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:58.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-21", "mode": "writeback"}]: dispatch 2024-02-14T17:49:58.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:58.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:58 smithi100 ceph-mon[39557]: pgmap v259: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:49:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi100-79590-31"}]': finished 2024-02-14T17:49:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:49:58.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-21"}]': finished 2024-02-14T17:49:58.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:58 smithi179 ceph-mon[33633]: osdmap e198: 8 total, 8 up, 8 in 2024-02-14T17:49:58.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-21", "mode": "writeback"}]: dispatch 2024-02-14T17:49:58.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1084699948' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:58.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-21", "mode": "writeback"}]: dispatch 2024-02-14T17:49:58.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]: dispatch 2024-02-14T17:49:58.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:58 smithi179 ceph-mon[33633]: pgmap v259: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:49:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-21", "mode": "writeback"}]': finished 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]': finished 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: osdmap e199: 8 total, 8 up, 8 in 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/755899464' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-79590-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-79590-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-21", "mode": "writeback"}]': finished 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]': finished 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: osdmap e199: 8 total, 8 up, 8 in 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/755899464' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-79590-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-79590-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:59.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:49:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-21", "mode": "writeback"}]': finished 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi100-79590-31"}]': finished 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: osdmap e199: 8 total, 8 up, 8 in 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/755899464' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-79590-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:59.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-79590-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:49:59.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:49:59.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:49:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:00.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:50:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/755899464' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-79590-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: osdmap e200: 8 total, 8 up, 8 in 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-79590-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-79590-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: pgmap v262: 420 pgs: 64 unknown, 356 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 4 pool(s) do not have an application enabled 2024-02-14T17:50:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-14T17:50:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: pool 'test-rados-api-smithi100-79838-21' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-14T17:50:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-02-14T17:50:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T17:50:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: application not enabled on pool 'WatchNotifysmithi100-80894-1' 2024-02-14T17:50:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: application not enabled on pool 'poolrace.1804289383' 2024-02-14T17:50:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: application not enabled on pool 'poolrace2.846930886' 2024-02-14T17:50:00.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[30138]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T17:50:00.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/755899464' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:00.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-79590-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:00.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: osdmap e200: 8 total, 8 up, 8 in 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-79590-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-79590-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: pgmap v262: 420 pgs: 64 unknown, 356 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 4 pool(s) do not have an application enabled 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: pool 'test-rados-api-smithi100-79838-21' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: application not enabled on pool 'WatchNotifysmithi100-80894-1' 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: application not enabled on pool 'poolrace.1804289383' 2024-02-14T17:50:00.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: application not enabled on pool 'poolrace2.846930886' 2024-02-14T17:50:00.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:00 smithi100 ceph-mon[39557]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T17:50:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/755899464' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi100-79612-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi100-79590-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: osdmap e200: 8 total, 8 up, 8 in 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-79590-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-79590-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21"}]: dispatch 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: pgmap v262: 420 pgs: 64 unknown, 356 active+clean; 584 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 4 pool(s) do not have an application enabled 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: pool 'test-rados-api-smithi100-79838-21' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-02-14T17:50:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T17:50:00.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: application not enabled on pool 'WatchNotifysmithi100-80894-1' 2024-02-14T17:50:00.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: application not enabled on pool 'poolrace.1804289383' 2024-02-14T17:50:00.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: application not enabled on pool 'poolrace2.846930886' 2024-02-14T17:50:00.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:00 smithi179 ceph-mon[33633]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T17:50:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:01 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21"}]': finished 2024-02-14T17:50:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:01 smithi100 ceph-mon[30138]: osdmap e201: 8 total, 8 up, 8 in 2024-02-14T17:50:01.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:01 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:01.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21"}]': finished 2024-02-14T17:50:01.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:01 smithi100 ceph-mon[39557]: osdmap e201: 8 total, 8 up, 8 in 2024-02-14T17:50:01.776 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:01 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:01.776 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-21"}]': finished 2024-02-14T17:50:01.776 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:01 smithi179 ceph-mon[33633]: osdmap e201: 8 total, 8 up, 8 in 2024-02-14T17:50:02.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:50:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:50:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-79590-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-79590-32"}]': finished 2024-02-14T17:50:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:02 smithi100 ceph-mon[30138]: osdmap e202: 8 total, 8 up, 8 in 2024-02-14T17:50:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:02 smithi100 ceph-mon[30138]: pgmap v265: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:50:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:02 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-79590-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-79590-32"}]': finished 2024-02-14T17:50:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:02 smithi100 ceph-mon[39557]: osdmap e202: 8 total, 8 up, 8 in 2024-02-14T17:50:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:02 smithi100 ceph-mon[39557]: pgmap v265: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:50:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:02 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi100-79590-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi100-79590-32"}]': finished 2024-02-14T17:50:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:02 smithi179 ceph-mon[33633]: osdmap e202: 8 total, 8 up, 8 in 2024-02-14T17:50:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:02 smithi179 ceph-mon[33633]: pgmap v265: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:50:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:02 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:03 smithi100 ceph-mon[30138]: osdmap e203: 8 total, 8 up, 8 in 2024-02-14T17:50:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:03 smithi100 ceph-mon[39557]: osdmap e203: 8 total, 8 up, 8 in 2024-02-14T17:50:03.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:03 smithi179 ceph-mon[33633]: osdmap e203: 8 total, 8 up, 8 in 2024-02-14T17:50:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: osdmap e204: 8 total, 8 up, 8 in 2024-02-14T17:50:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2522710407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: pgmap v268: 356 pgs: 64 unknown, 292 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]': finished 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: osdmap e205: 8 total, 8 up, 8 in 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: osdmap e204: 8 total, 8 up, 8 in 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2522710407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:04.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:04.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: pgmap v268: 356 pgs: 64 unknown, 292 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:04.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]': finished 2024-02-14T17:50:04.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:04.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: osdmap e205: 8 total, 8 up, 8 in 2024-02-14T17:50:04.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: osdmap e204: 8 total, 8 up, 8 in 2024-02-14T17:50:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2522710407' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: pgmap v268: 356 pgs: 64 unknown, 292 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:50:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi100-79590-32"}]': finished 2024-02-14T17:50:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2058949997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: osdmap e205: 8 total, 8 up, 8 in 2024-02-14T17:50:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]: dispatch 2024-02-14T17:50:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]': finished 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[30138]: osdmap e206: 8 total, 8 up, 8 in 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/937942808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]': finished 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:05.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[39557]: osdmap e206: 8 total, 8 up, 8 in 2024-02-14T17:50:05.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:05.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/937942808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:05.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:05.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:05.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi100-79590-32"}]': finished 2024-02-14T17:50:05.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:05.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:05 smithi179 ceph-mon[33633]: osdmap e206: 8 total, 8 up, 8 in 2024-02-14T17:50:05.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:05.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/937942808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:05.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:05.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-79590-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-79590-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: pgmap v271: 388 pgs: 96 unknown, 292 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-23"}]': finished 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-79590-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: osdmap e207: 8 total, 8 up, 8 in 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-23", "mode": "writeback"}]: dispatch 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-79590-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-23", "mode": "writeback"}]: dispatch 2024-02-14T17:50:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-79590-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-79590-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-79590-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: pgmap v271: 388 pgs: 96 unknown, 292 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-23"}]': finished 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-79590-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: osdmap e207: 8 total, 8 up, 8 in 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-23", "mode": "writeback"}]: dispatch 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-79590-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-23", "mode": "writeback"}]: dispatch 2024-02-14T17:50:06.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-79590-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-79590-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-79590-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: pgmap v271: 388 pgs: 96 unknown, 292 active+clean; 583 KiB data, 574 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-23"}]': finished 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi100-79590-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: osdmap e207: 8 total, 8 up, 8 in 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-23", "mode": "writeback"}]: dispatch 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-79590-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:06.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-23", "mode": "writeback"}]: dispatch 2024-02-14T17:50:06.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-79590-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:07.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:07 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:07.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-23", "mode": "writeback"}]': finished 2024-02-14T17:50:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:07 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-23", "mode": "writeback"}]': finished 2024-02-14T17:50:08.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:07 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:08.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-23", "mode": "writeback"}]': finished 2024-02-14T17:50:09.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:08 smithi179 ceph-mon[33633]: pgmap v274: 420 pgs: 32 unknown, 32 creating+peering, 356 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:50:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:08 smithi179 ceph-mon[33633]: osdmap e208: 8 total, 8 up, 8 in 2024-02-14T17:50:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3454496512' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:08 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-79590-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-79590-33"}]': finished 2024-02-14T17:50:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3454496512' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:08 smithi179 ceph-mon[33633]: osdmap e209: 8 total, 8 up, 8 in 2024-02-14T17:50:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[30138]: pgmap v274: 420 pgs: 32 unknown, 32 creating+peering, 356 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:50:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[30138]: osdmap e208: 8 total, 8 up, 8 in 2024-02-14T17:50:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3454496512' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-79590-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-79590-33"}]': finished 2024-02-14T17:50:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3454496512' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:09.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[30138]: osdmap e209: 8 total, 8 up, 8 in 2024-02-14T17:50:09.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:09.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[39557]: pgmap v274: 420 pgs: 32 unknown, 32 creating+peering, 356 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:50:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[39557]: osdmap e208: 8 total, 8 up, 8 in 2024-02-14T17:50:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3454496512' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi100-79590-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi100-79590-33"}]': finished 2024-02-14T17:50:09.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3454496512' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:09.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[39557]: osdmap e209: 8 total, 8 up, 8 in 2024-02-14T17:50:09.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:09.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:10.450 INFO:tasks.workunit.client.0.smithi100.stdout:d 0x5648c476be50 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476bf60 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476c070 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476c790 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476c2f0 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476c400 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476c510 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476c660 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476cd40 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476c8a0 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476c9b0 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476cac0 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476cbd0 2024-02-14T17:50:10.451 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476d330 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476ce50 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476cfb0 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476d0c0 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476d440 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476d570 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476db40 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476d6a0 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476d7b0 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476d8c0 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476d9f0 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476e0f0 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476dc50 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476dda0 2024-02-14T17:50:10.452 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476deb0 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476dfc0 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476e6a0 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476e200 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476e310 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476e440 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476e550 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476ec50 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476e7b0 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476e900 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476ea10 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476ed60 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476ee70 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476f420 2024-02-14T17:50:10.453 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476ef80 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476f0b0 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476f1c0 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476f310 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476f9d0 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476f530 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476f640 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476f770 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476f880 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476ffe0 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476fb40 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476fc50 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476fda0 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c476feb0 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4770590 2024-02-14T17:50:10.454 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47700f0 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4770220 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4770330 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4770460 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4770b40 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47706a0 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47707d0 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47708e0 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4770a10 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47710f0 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4770c50 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4770d60 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4770e70 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4770f80 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47716c0 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4771220 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4771350 2024-02-14T17:50:10.455 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4771480 2024-02-14T17:50:10.456 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47717d0 2024-02-14T17:50:10.456 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4771c60 2024-02-14T17:50:10.456 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4771d70 2024-02-14T17:50:10.456 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47718e0 2024-02-14T17:50:10.456 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47719f0 2024-02-14T17:50:10.456 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4771b20 2024-02-14T17:50:10.456 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4771ea0 2024-02-14T17:50:10.456 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4771fd0 2024-02-14T17:50:10.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:50:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:10.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:10.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[39557]: osdmap e210: 8 total, 8 up, 8 in 2024-02-14T17:50:10.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1949983642' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:10.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:10.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:10.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[39557]: pgmap v277: 460 pgs: 72 unknown, 32 creating+peering, 356 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:50:10.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:10.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[30138]: osdmap e210: 8 total, 8 up, 8 in 2024-02-14T17:50:10.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1949983642' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:10.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:10.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:10.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:10 smithi100 ceph-mon[30138]: pgmap v277: 460 pgs: 72 unknown, 32 creating+peering, 356 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:50:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:10 smithi179 ceph-mon[33633]: osdmap e210: 8 total, 8 up, 8 in 2024-02-14T17:50:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1949983642' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:10.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23"}]: dispatch 2024-02-14T17:50:10.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:10 smithi179 ceph-mon[33633]: pgmap v277: 460 pgs: 72 unknown, 32 creating+peering, 356 active+clean; 584 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:50:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1949983642' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23"}]': finished 2024-02-14T17:50:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[39557]: osdmap e211: 8 total, 8 up, 8 in 2024-02-14T17:50:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:11.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:11.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1949983642' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:11.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23"}]': finished 2024-02-14T17:50:11.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[30138]: osdmap e211: 8 total, 8 up, 8 in 2024-02-14T17:50:11.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:11.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:11.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:11.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:11 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:11.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1949983642' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:11.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-23"}]': finished 2024-02-14T17:50:11.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:11 smithi179 ceph-mon[33633]: osdmap e211: 8 total, 8 up, 8 in 2024-02-14T17:50:11.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:11.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:11.788 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:12.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:50:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:50:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]': finished 2024-02-14T17:50:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:12 smithi179 ceph-mon[33633]: osdmap e212: 8 total, 8 up, 8 in 2024-02-14T17:50:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2120831814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:12 smithi179 ceph-mon[33633]: pgmap v280: 452 pgs: 32 unknown, 420 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:50:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:12 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]': finished 2024-02-14T17:50:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[30138]: osdmap e212: 8 total, 8 up, 8 in 2024-02-14T17:50:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2120831814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[30138]: pgmap v280: 452 pgs: 32 unknown, 420 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:50:12.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:12.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi100-79590-33"}]': finished 2024-02-14T17:50:12.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/15844432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:12.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[39557]: osdmap e212: 8 total, 8 up, 8 in 2024-02-14T17:50:12.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2120831814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]: dispatch 2024-02-14T17:50:12.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[39557]: pgmap v280: 452 pgs: 32 unknown, 420 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:50:12.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:12.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:12 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]': finished 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: osdmap e213: 8 total, 8 up, 8 in 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-79590-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-79590-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: osdmap e214: 8 total, 8 up, 8 in 2024-02-14T17:50:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-79590-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]': finished 2024-02-14T17:50:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: osdmap e213: 8 total, 8 up, 8 in 2024-02-14T17:50:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-79590-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-79590-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: osdmap e214: 8 total, 8 up, 8 in 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-79590-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi100-79590-33"}]': finished 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi100-79612-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: osdmap e213: 8 total, 8 up, 8 in 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:13.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:13.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-79590-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:13.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:13.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi100-79590-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:13.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: osdmap e214: 8 total, 8 up, 8 in 2024-02-14T17:50:13.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-79590-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:14 smithi179 ceph-mon[33633]: pgmap v283: 452 pgs: 32 unknown, 420 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:50:14.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:14 smithi100 ceph-mon[30138]: pgmap v283: 452 pgs: 32 unknown, 420 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:50:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:14 smithi100 ceph-mon[39557]: pgmap v283: 452 pgs: 32 unknown, 420 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:50:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:15 smithi179 ceph-mon[33633]: osdmap e215: 8 total, 8 up, 8 in 2024-02-14T17:50:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:15 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-79590-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-79590-34"}]': finished 2024-02-14T17:50:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:15 smithi179 ceph-mon[33633]: osdmap e216: 8 total, 8 up, 8 in 2024-02-14T17:50:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[30138]: osdmap e215: 8 total, 8 up, 8 in 2024-02-14T17:50:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-79590-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-79590-34"}]': finished 2024-02-14T17:50:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[30138]: osdmap e216: 8 total, 8 up, 8 in 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[39557]: osdmap e215: 8 total, 8 up, 8 in 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi100-79590-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi100-79590-34"}]': finished 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[39557]: osdmap e216: 8 total, 8 up, 8 in 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:15.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:16 smithi179 ceph-mon[33633]: pgmap v286: 396 pgs: 40 unknown, 356 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-25"}]': finished 2024-02-14T17:50:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:16 smithi179 ceph-mon[33633]: osdmap e217: 8 total, 8 up, 8 in 2024-02-14T17:50:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-25", "mode": "writeback"}]: dispatch 2024-02-14T17:50:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-25", "mode": "writeback"}]: dispatch 2024-02-14T17:50:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:16 smithi100 ceph-mon[30138]: pgmap v286: 396 pgs: 40 unknown, 356 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-25"}]': finished 2024-02-14T17:50:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:16 smithi100 ceph-mon[30138]: osdmap e217: 8 total, 8 up, 8 in 2024-02-14T17:50:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-25", "mode": "writeback"}]: dispatch 2024-02-14T17:50:16.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-25", "mode": "writeback"}]: dispatch 2024-02-14T17:50:16.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:16 smithi100 ceph-mon[39557]: pgmap v286: 396 pgs: 40 unknown, 356 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:16.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-25"}]': finished 2024-02-14T17:50:16.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:16 smithi100 ceph-mon[39557]: osdmap e217: 8 total, 8 up, 8 in 2024-02-14T17:50:16.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-25", "mode": "writeback"}]: dispatch 2024-02-14T17:50:16.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-25", "mode": "writeback"}]: dispatch 2024-02-14T17:50:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:17 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-25", "mode": "writeback"}]': finished 2024-02-14T17:50:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:17 smithi179 ceph-mon[33633]: osdmap e218: 8 total, 8 up, 8 in 2024-02-14T17:50:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:17 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-25", "mode": "writeback"}]': finished 2024-02-14T17:50:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:17 smithi100 ceph-mon[39557]: osdmap e218: 8 total, 8 up, 8 in 2024-02-14T17:50:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:17 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-25", "mode": "writeback"}]': finished 2024-02-14T17:50:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:17 smithi100 ceph-mon[30138]: osdmap e218: 8 total, 8 up, 8 in 2024-02-14T17:50:17.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: pgmap v289: 324 pgs: 324 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-79590-34"}]': finished 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: osdmap e219: 8 total, 8 up, 8 in 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4067869785' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-79612-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-79612-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: pgmap v289: 324 pgs: 324 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-79590-34"}]': finished 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: osdmap e219: 8 total, 8 up, 8 in 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4067869785' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-79612-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-79612-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: pgmap v289: 324 pgs: 324 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi100-79590-34"}]': finished 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: osdmap e219: 8 total, 8 up, 8 in 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-79590-34"}]: dispatch 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25"}]: dispatch 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4067869785' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-79612-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:18.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-79612-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:19.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:50:19.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-79590-34"}]': finished 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25"}]': finished 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-79612-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: osdmap e220: 8 total, 8 up, 8 in 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-79590-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:19.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-79590-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:19.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:50:19.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:50:19.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:19.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:19.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-79590-34"}]': finished 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25"}]': finished 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-79612-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: osdmap e220: 8 total, 8 up, 8 in 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-79590-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-79590-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:50:19.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1341738432' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi100-79590-34"}]': finished 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-25"}]': finished 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi100-79612-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: osdmap e220: 8 total, 8 up, 8 in 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-79590-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:19.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-79590-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:20.786 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:50:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:20 smithi179 ceph-mon[33633]: pgmap v292: 356 pgs: 32 unknown, 324 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-79590-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:20 smithi179 ceph-mon[33633]: osdmap e221: 8 total, 8 up, 8 in 2024-02-14T17:50:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-79590-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-79590-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:21.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-mon[30138]: pgmap v292: 356 pgs: 32 unknown, 324 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:21.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-79590-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:21.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-mon[30138]: osdmap e221: 8 total, 8 up, 8 in 2024-02-14T17:50:21.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-79590-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:21.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-79590-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:21.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-mon[39557]: pgmap v292: 356 pgs: 32 unknown, 324 active+clean; 583 KiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:21.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi100-79590-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:21.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-mon[39557]: osdmap e221: 8 total, 8 up, 8 in 2024-02-14T17:50:21.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-79590-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:21.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi100-79590-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:22.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:50:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:50:22.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:22 smithi179 ceph-mon[33633]: osdmap e222: 8 total, 8 up, 8 in 2024-02-14T17:50:22.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3340080674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-79612-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-79612-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:22 smithi179 ceph-mon[33633]: pgmap v295: 356 pgs: 8 creating+peering, 56 unknown, 292 active+clean; 583 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:22 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[30138]: osdmap e222: 8 total, 8 up, 8 in 2024-02-14T17:50:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3340080674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-79612-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-79612-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[30138]: pgmap v295: 356 pgs: 8 creating+peering, 56 unknown, 292 active+clean; 583 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[39557]: osdmap e222: 8 total, 8 up, 8 in 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3340080674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-79612-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-79612-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[39557]: pgmap v295: 356 pgs: 8 creating+peering, 56 unknown, 292 active+clean; 583 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:22.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:22 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:23 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:23 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi100-79590-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-79590-35"}]': finished 2024-02-14T17:50:23.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:23 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:23.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:23 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-79612-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:23.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:23 smithi179 ceph-mon[33633]: osdmap e223: 8 total, 8 up, 8 in 2024-02-14T17:50:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:23 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:23 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi100-79590-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-79590-35"}]': finished 2024-02-14T17:50:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:23 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:23 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-79612-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:23 smithi100 ceph-mon[39557]: osdmap e223: 8 total, 8 up, 8 in 2024-02-14T17:50:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:23 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:23 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi100-79590-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi100-79590-35"}]': finished 2024-02-14T17:50:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:23 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:23 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi100-79612-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:23.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:23 smithi100 ceph-mon[30138]: osdmap e223: 8 total, 8 up, 8 in 2024-02-14T17:50:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[39557]: osdmap e224: 8 total, 8 up, 8 in 2024-02-14T17:50:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[39557]: pgmap v298: 332 pgs: 5 creating+peering, 35 unknown, 292 active+clean; 583 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[30138]: osdmap e224: 8 total, 8 up, 8 in 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:24 smithi100 ceph-mon[30138]: pgmap v298: 332 pgs: 5 creating+peering, 35 unknown, 292 active+clean; 583 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:24 smithi179 ceph-mon[33633]: osdmap e224: 8 total, 8 up, 8 in 2024-02-14T17:50:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:24 smithi179 ceph-mon[33633]: pgmap v298: 332 pgs: 5 creating+peering, 35 unknown, 292 active+clean; 583 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[30138]: osdmap e225: 8 total, 8 up, 8 in 2024-02-14T17:50:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[39557]: osdmap e225: 8 total, 8 up, 8 in 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:25.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:25.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:25.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:25.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:25.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:25 smithi179 ceph-mon[33633]: osdmap e225: 8 total, 8 up, 8 in 2024-02-14T17:50:25.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:25.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:25.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:25.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:25.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]': finished 2024-02-14T17:50:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-27"}]': finished 2024-02-14T17:50:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-27", "mode": "writeback"}]: dispatch 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[30138]: osdmap e226: 8 total, 8 up, 8 in 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-27", "mode": "writeback"}]: dispatch 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[30138]: pgmap v301: 324 pgs: 5 creating+peering, 27 unknown, 292 active+clean; 583 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]': finished 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-27"}]': finished 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-27", "mode": "writeback"}]: dispatch 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[39557]: osdmap e226: 8 total, 8 up, 8 in 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-27", "mode": "writeback"}]: dispatch 2024-02-14T17:50:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[39557]: pgmap v301: 324 pgs: 5 creating+peering, 27 unknown, 292 active+clean; 583 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:26.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:26 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi100-79590-35"}]': finished 2024-02-14T17:50:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-27"}]': finished 2024-02-14T17:50:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/580720761' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-27", "mode": "writeback"}]: dispatch 2024-02-14T17:50:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:26 smithi179 ceph-mon[33633]: osdmap e226: 8 total, 8 up, 8 in 2024-02-14T17:50:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]: dispatch 2024-02-14T17:50:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-27", "mode": "writeback"}]: dispatch 2024-02-14T17:50:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:26 smithi179 ceph-mon[33633]: pgmap v301: 324 pgs: 5 creating+peering, 27 unknown, 292 active+clean; 583 KiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:26 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:27.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:27.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-46"}]': finished 2024-02-14T17:50:27.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]': finished 2024-02-14T17:50:27.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-27", "mode": "writeback"}]': finished 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: osdmap e227: 8 total, 8 up, 8 in 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-79590-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-79590-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-46"}]': finished 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]': finished 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-27", "mode": "writeback"}]': finished 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: osdmap e227: 8 total, 8 up, 8 in 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-79590-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:27.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-79590-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:27.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:27.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-46"}]': finished 2024-02-14T17:50:27.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi100-79590-35"}]': finished 2024-02-14T17:50:27.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-27", "mode": "writeback"}]': finished 2024-02-14T17:50:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: osdmap e227: 8 total, 8 up, 8 in 2024-02-14T17:50:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-79590-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-79590-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-79590-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[30138]: osdmap e228: 8 total, 8 up, 8 in 2024-02-14T17:50:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-79590-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-79590-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[30138]: pgmap v304: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:28.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-79590-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[39557]: osdmap e228: 8 total, 8 up, 8 in 2024-02-14T17:50:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-79590-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-79590-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[39557]: pgmap v304: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:28.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:28 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:28.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi100-79590-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:28.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:28 smithi179 ceph-mon[33633]: osdmap e228: 8 total, 8 up, 8 in 2024-02-14T17:50:28.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-79590-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:28.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-79590-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:28.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:28.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:28.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:28 smithi179 ceph-mon[33633]: pgmap v304: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:28.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:28 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[39557]: osdmap e229: 8 total, 8 up, 8 in 2024-02-14T17:50:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:29.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:29.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:29.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[30138]: osdmap e229: 8 total, 8 up, 8 in 2024-02-14T17:50:29.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:29.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:29.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:29.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:29.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:29.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:29 smithi179 ceph-mon[33633]: osdmap e229: 8 total, 8 up, 8 in 2024-02-14T17:50:29.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27"}]: dispatch 2024-02-14T17:50:29.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:29.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:30.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:50:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:30.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-79590-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-79590-36"}]': finished 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27"}]': finished 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]': finished 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[30138]: osdmap e230: 8 total, 8 up, 8 in 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[30138]: pgmap v307: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-79590-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-79590-36"}]': finished 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27"}]': finished 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]': finished 2024-02-14T17:50:30.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[39557]: osdmap e230: 8 total, 8 up, 8 in 2024-02-14T17:50:30.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:30.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:30.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:30 smithi100 ceph-mon[39557]: pgmap v307: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:30.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:30 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:30.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi100-79590-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi100-79590-36"}]': finished 2024-02-14T17:50:30.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-27"}]': finished 2024-02-14T17:50:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-46"}]': finished 2024-02-14T17:50:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:30 smithi179 ceph-mon[33633]: osdmap e230: 8 total, 8 up, 8 in 2024-02-14T17:50:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1534644433' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]: dispatch 2024-02-14T17:50:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:30 smithi179 ceph-mon[33633]: pgmap v307: 332 pgs: 8 unknown, 324 active+clean; 584 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:50:31.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]': finished 2024-02-14T17:50:31.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[30138]: osdmap e231: 8 total, 8 up, 8 in 2024-02-14T17:50:31.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]': finished 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[39557]: osdmap e231: 8 total, 8 up, 8 in 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:31.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:31.806 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-46"}]': finished 2024-02-14T17:50:31.806 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:31 smithi179 ceph-mon[33633]: osdmap e231: 8 total, 8 up, 8 in 2024-02-14T17:50:31.807 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.807 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.807 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.807 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:31.807 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:31.807 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:32.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:50:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:50:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[30138]: osdmap e232: 8 total, 8 up, 8 in 2024-02-14T17:50:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[30138]: pgmap v310: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[39557]: osdmap e232: 8 total, 8 up, 8 in 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:32.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:32.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:32.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[39557]: pgmap v310: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:32.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:32 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:32.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi100-79612-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:32.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:32 smithi179 ceph-mon[33633]: osdmap e232: 8 total, 8 up, 8 in 2024-02-14T17:50:32.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:32.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:32.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:32.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:32.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:32.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:32.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:32 smithi179 ceph-mon[33633]: pgmap v310: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:32.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:32 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]': finished 2024-02-14T17:50:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: osdmap e233: 8 total, 8 up, 8 in 2024-02-14T17:50:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-47"}]': finished 2024-02-14T17:50:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]': finished 2024-02-14T17:50:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: osdmap e234: 8 total, 8 up, 8 in 2024-02-14T17:50:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]': finished 2024-02-14T17:50:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: osdmap e233: 8 total, 8 up, 8 in 2024-02-14T17:50:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-47"}]': finished 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]': finished 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: osdmap e234: 8 total, 8 up, 8 in 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi100-79590-36"}]': finished 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2881400684' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: osdmap e233: 8 total, 8 up, 8 in 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]: dispatch 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi100-79612-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi100-79612-47"}]': finished 2024-02-14T17:50:33.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi100-79590-36"}]': finished 2024-02-14T17:50:33.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: osdmap e234: 8 total, 8 up, 8 in 2024-02-14T17:50:33.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:33.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:33.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[30138]: pgmap v313: 332 pgs: 40 unknown, 292 active+clean; 584 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[30138]: osdmap e235: 8 total, 8 up, 8 in 2024-02-14T17:50:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-79590-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:34.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[39557]: pgmap v313: 332 pgs: 40 unknown, 292 active+clean; 584 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:34.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:34.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[39557]: osdmap e235: 8 total, 8 up, 8 in 2024-02-14T17:50:34.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-79590-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:34.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:34.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:34 smithi179 ceph-mon[33633]: pgmap v313: 332 pgs: 40 unknown, 292 active+clean; 584 KiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:34.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:34 smithi179 ceph-mon[33633]: osdmap e235: 8 total, 8 up, 8 in 2024-02-14T17:50:34.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-79590-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:34.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:34.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:36 smithi179 ceph-mon[33633]: osdmap e236: 8 total, 8 up, 8 in 2024-02-14T17:50:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:36 smithi179 ceph-mon[33633]: pgmap v316: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[30138]: osdmap e236: 8 total, 8 up, 8 in 2024-02-14T17:50:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:36.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[30138]: pgmap v316: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[39557]: osdmap e236: 8 total, 8 up, 8 in 2024-02-14T17:50:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:36.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:36 smithi100 ceph-mon[39557]: pgmap v316: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-79590-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-79590-37"}]': finished 2024-02-14T17:50:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-29"}]': finished 2024-02-14T17:50:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]': finished 2024-02-14T17:50:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:37 smithi179 ceph-mon[33633]: osdmap e237: 8 total, 8 up, 8 in 2024-02-14T17:50:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-29", "mode": "writeback"}]: dispatch 2024-02-14T17:50:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-29", "mode": "writeback"}]: dispatch 2024-02-14T17:50:37.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-79590-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-79590-37"}]': finished 2024-02-14T17:50:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-29"}]': finished 2024-02-14T17:50:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]': finished 2024-02-14T17:50:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[30138]: osdmap e237: 8 total, 8 up, 8 in 2024-02-14T17:50:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-29", "mode": "writeback"}]: dispatch 2024-02-14T17:50:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-29", "mode": "writeback"}]: dispatch 2024-02-14T17:50:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi100-79590-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi100-79590-37"}]': finished 2024-02-14T17:50:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-29"}]': finished 2024-02-14T17:50:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi100-79612-47"}]': finished 2024-02-14T17:50:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[39557]: osdmap e237: 8 total, 8 up, 8 in 2024-02-14T17:50:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-29", "mode": "writeback"}]: dispatch 2024-02-14T17:50:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2497570091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-29", "mode": "writeback"}]: dispatch 2024-02-14T17:50:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]: dispatch 2024-02-14T17:50:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:38 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-29", "mode": "writeback"}]': finished 2024-02-14T17:50:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]': finished 2024-02-14T17:50:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:38 smithi179 ceph-mon[33633]: osdmap e238: 8 total, 8 up, 8 in 2024-02-14T17:50:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-79612-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:38 smithi179 ceph-mon[33633]: pgmap v319: 332 pgs: 3 creating+peering, 5 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-14T17:50:38.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:38 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-29", "mode": "writeback"}]': finished 2024-02-14T17:50:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]': finished 2024-02-14T17:50:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[30138]: osdmap e238: 8 total, 8 up, 8 in 2024-02-14T17:50:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-79612-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[30138]: pgmap v319: 332 pgs: 3 creating+peering, 5 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-14T17:50:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-29", "mode": "writeback"}]': finished 2024-02-14T17:50:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi100-79612-47"}]': finished 2024-02-14T17:50:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[39557]: osdmap e238: 8 total, 8 up, 8 in 2024-02-14T17:50:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-79612-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[39557]: pgmap v319: 332 pgs: 3 creating+peering, 5 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-14T17:50:38.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:38 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:39.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-79612-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:39.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:39 smithi179 ceph-mon[33633]: osdmap e239: 8 total, 8 up, 8 in 2024-02-14T17:50:39.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-79612-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:39.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:39.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-79612-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:39.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:39 smithi100 ceph-mon[30138]: osdmap e239: 8 total, 8 up, 8 in 2024-02-14T17:50:39.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-79612-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:39.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:39.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi100-79612-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:39.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:39 smithi100 ceph-mon[39557]: osdmap e239: 8 total, 8 up, 8 in 2024-02-14T17:50:39.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-79612-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:39.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:40.802 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:50:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37"}]': finished 2024-02-14T17:50:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: osdmap e240: 8 total, 8 up, 8 in 2024-02-14T17:50:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: pgmap v322: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-14T17:50:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-79612-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-79612-48"}]': finished 2024-02-14T17:50:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-79590-37"}]': finished 2024-02-14T17:50:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: osdmap e241: 8 total, 8 up, 8 in 2024-02-14T17:50:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:40.803 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-79590-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:40.803 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37"}]': finished 2024-02-14T17:50:40.803 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: osdmap e240: 8 total, 8 up, 8 in 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: pgmap v322: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-79612-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-79612-48"}]': finished 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-79590-37"}]': finished 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: osdmap e241: 8 total, 8 up, 8 in 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:40.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-79590-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi100-79590-37"}]': finished 2024-02-14T17:50:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: osdmap e240: 8 total, 8 up, 8 in 2024-02-14T17:50:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-79590-37"}]: dispatch 2024-02-14T17:50:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: pgmap v322: 324 pgs: 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi100-79612-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi100-79612-48"}]': finished 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2552100436' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi100-79590-37"}]': finished 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: osdmap e241: 8 total, 8 up, 8 in 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29"}]: dispatch 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-79590-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:42.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:50:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:50:42.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:42 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:42 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29"}]': finished 2024-02-14T17:50:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-79590-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:42 smithi179 ceph-mon[33633]: osdmap e242: 8 total, 8 up, 8 in 2024-02-14T17:50:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-79590-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:42 smithi179 ceph-mon[33633]: pgmap v325: 332 pgs: 3 creating+activating, 5 creating+peering, 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-14T17:50:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:42 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29"}]': finished 2024-02-14T17:50:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-79590-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[30138]: osdmap e242: 8 total, 8 up, 8 in 2024-02-14T17:50:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-79590-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[30138]: pgmap v325: 332 pgs: 3 creating+activating, 5 creating+peering, 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-14T17:50:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:50:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-29"}]': finished 2024-02-14T17:50:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi100-79590-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[39557]: osdmap e242: 8 total, 8 up, 8 in 2024-02-14T17:50:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-79590-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[39557]: pgmap v325: 332 pgs: 3 creating+activating, 5 creating+peering, 324 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-14T17:50:42.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:42 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:43 smithi179 ceph-mon[33633]: osdmap e243: 8 total, 8 up, 8 in 2024-02-14T17:50:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:43 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-79590-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-79590-38"}]': finished 2024-02-14T17:50:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-79612-48"}]': finished 2024-02-14T17:50:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:43 smithi179 ceph-mon[33633]: osdmap e244: 8 total, 8 up, 8 in 2024-02-14T17:50:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[30138]: osdmap e243: 8 total, 8 up, 8 in 2024-02-14T17:50:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-79590-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-79590-38"}]': finished 2024-02-14T17:50:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-79612-48"}]': finished 2024-02-14T17:50:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[30138]: osdmap e244: 8 total, 8 up, 8 in 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[39557]: osdmap e243: 8 total, 8 up, 8 in 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi100-79590-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi100-79590-38"}]': finished 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi100-79612-48"}]': finished 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[39557]: osdmap e244: 8 total, 8 up, 8 in 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-79612-48"}]: dispatch 2024-02-14T17:50:43.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[39557]: pgmap v328: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-14T17:50:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-79612-48"}]': finished 2024-02-14T17:50:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[39557]: osdmap e245: 8 total, 8 up, 8 in 2024-02-14T17:50:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[30138]: pgmap v328: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-14T17:50:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-79612-48"}]': finished 2024-02-14T17:50:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[30138]: osdmap e245: 8 total, 8 up, 8 in 2024-02-14T17:50:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:44.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:44 smithi179 ceph-mon[33633]: pgmap v328: 332 pgs: 40 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-14T17:50:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3462405716' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi100-79612-48"}]': finished 2024-02-14T17:50:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:44 smithi179 ceph-mon[33633]: osdmap e245: 8 total, 8 up, 8 in 2024-02-14T17:50:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-79612-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-79612-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[39557]: osdmap e246: 8 total, 8 up, 8 in 2024-02-14T17:50:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-79612-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-79612-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-79612-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[30138]: osdmap e246: 8 total, 8 up, 8 in 2024-02-14T17:50:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-79612-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:45.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:45.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-79612-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi100-79612-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:45 smithi179 ceph-mon[33633]: osdmap e246: 8 total, 8 up, 8 in 2024-02-14T17:50:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-79612-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:45.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:46.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:46 smithi179 ceph-mon[33633]: pgmap v331: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:46.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-31"}]': finished 2024-02-14T17:50:46.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:46 smithi100 ceph-mon[30138]: pgmap v331: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:46.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-31"}]': finished 2024-02-14T17:50:46.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:46 smithi100 ceph-mon[39557]: pgmap v331: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:46.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-31"}]': finished 2024-02-14T17:50:47.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-79590-38"}]': finished 2024-02-14T17:50:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-31", "mode": "writeback"}]: dispatch 2024-02-14T17:50:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: osdmap e247: 8 total, 8 up, 8 in 2024-02-14T17:50:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-31", "mode": "writeback"}]: dispatch 2024-02-14T17:50:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-79612-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-79612-49"}]': finished 2024-02-14T17:50:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-79590-38"}]': finished 2024-02-14T17:50:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-31", "mode": "writeback"}]': finished 2024-02-14T17:50:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: osdmap e248: 8 total, 8 up, 8 in 2024-02-14T17:50:47.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-79590-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:47.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-79590-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:47.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:47.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-79590-38"}]': finished 2024-02-14T17:50:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-31", "mode": "writeback"}]: dispatch 2024-02-14T17:50:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: osdmap e247: 8 total, 8 up, 8 in 2024-02-14T17:50:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-31", "mode": "writeback"}]: dispatch 2024-02-14T17:50:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-79612-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-79612-49"}]': finished 2024-02-14T17:50:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-79590-38"}]': finished 2024-02-14T17:50:47.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-31", "mode": "writeback"}]': finished 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: osdmap e248: 8 total, 8 up, 8 in 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-79590-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-79590-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi100-79590-38"}]': finished 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-31", "mode": "writeback"}]: dispatch 2024-02-14T17:50:47.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: osdmap e247: 8 total, 8 up, 8 in 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-79590-38"}]: dispatch 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-31", "mode": "writeback"}]: dispatch 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi100-79612-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi100-79612-49"}]': finished 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1042761310' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi100-79590-38"}]': finished 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-31", "mode": "writeback"}]': finished 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: osdmap e248: 8 total, 8 up, 8 in 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:47.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-79590-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:47.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-79590-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:47.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:47.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:48 smithi179 ceph-mon[33633]: pgmap v334: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:48 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-79590-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:48 smithi179 ceph-mon[33633]: osdmap e249: 8 total, 8 up, 8 in 2024-02-14T17:50:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-79590-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-79590-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[30138]: pgmap v334: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-79590-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[30138]: osdmap e249: 8 total, 8 up, 8 in 2024-02-14T17:50:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-79590-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-79590-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[39557]: pgmap v334: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi100-79590-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[39557]: osdmap e249: 8 total, 8 up, 8 in 2024-02-14T17:50:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-79590-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31"}]: dispatch 2024-02-14T17:50:48.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-79590-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:49.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:49 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:49.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:49 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31"}]': finished 2024-02-14T17:50:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:49 smithi179 ceph-mon[33633]: osdmap e250: 8 total, 8 up, 8 in 2024-02-14T17:50:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:49.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:49 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:49.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:49 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31"}]': finished 2024-02-14T17:50:49.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:49 smithi100 ceph-mon[39557]: osdmap e250: 8 total, 8 up, 8 in 2024-02-14T17:50:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:49.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:49 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:49.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:49 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-31"}]': finished 2024-02-14T17:50:49.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:49 smithi100 ceph-mon[30138]: osdmap e250: 8 total, 8 up, 8 in 2024-02-14T17:50:49.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:50.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:50 smithi179 ceph-mon[33633]: pgmap v337: 324 pgs: 324 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:50.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-79590-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-79590-39"}]': finished 2024-02-14T17:50:50.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-79612-49"}]': finished 2024-02-14T17:50:50.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:50 smithi179 ceph-mon[33633]: osdmap e251: 8 total, 8 up, 8 in 2024-02-14T17:50:50.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:50:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:50:50.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-mon[39557]: pgmap v337: 324 pgs: 324 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:50.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-79590-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-79590-39"}]': finished 2024-02-14T17:50:50.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-79612-49"}]': finished 2024-02-14T17:50:50.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-mon[39557]: osdmap e251: 8 total, 8 up, 8 in 2024-02-14T17:50:50.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-mon[30138]: pgmap v337: 324 pgs: 324 active+clean; 4.6 MiB data, 588 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:50:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi100-79590-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi100-79590-39"}]': finished 2024-02-14T17:50:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi100-79612-49"}]': finished 2024-02-14T17:50:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-mon[30138]: osdmap e251: 8 total, 8 up, 8 in 2024-02-14T17:50:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-79612-49"}]: dispatch 2024-02-14T17:50:52.308 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:50:51 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:50:52.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-79612-49"}]': finished 2024-02-14T17:50:52.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: osdmap e252: 8 total, 8 up, 8 in 2024-02-14T17:50:52.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-79612-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-79612-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: pgmap v340: 332 pgs: 13 creating+peering, 26 unknown, 293 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:50:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:52 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-79612-49"}]': finished 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: osdmap e252: 8 total, 8 up, 8 in 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-79612-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-79612-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: pgmap v340: 332 pgs: 13 creating+peering, 26 unknown, 293 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:50:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2209119189' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi100-79612-49"}]': finished 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: osdmap e252: 8 total, 8 up, 8 in 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-79612-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-79612-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: pgmap v340: 332 pgs: 13 creating+peering, 26 unknown, 293 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:50:52.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:52 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:50:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-79612-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:53 smithi179 ceph-mon[33633]: osdmap e253: 8 total, 8 up, 8 in 2024-02-14T17:50:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-79612-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-79612-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:53.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:53 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:53.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:53.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-79612-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:53.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[30138]: osdmap e253: 8 total, 8 up, 8 in 2024-02-14T17:50:53.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-79612-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:53.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-79612-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:53.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:53.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:53.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:53.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:53.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:53.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:50:53.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi100-79612-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:53.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[39557]: osdmap e253: 8 total, 8 up, 8 in 2024-02-14T17:50:53.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-79612-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:53.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-79612-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:53.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:53.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:53.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:53.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:50:53.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:53 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:54.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]': finished 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: osdmap e254: 8 total, 8 up, 8 in 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: pgmap v343: 324 pgs: 6 creating+peering, 26 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-79612-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-79612-50"}]': finished 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]': finished 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-33"}]': finished 2024-02-14T17:50:54.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: osdmap e255: 8 total, 8 up, 8 in 2024-02-14T17:50:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-33", "mode": "writeback"}]: dispatch 2024-02-14T17:50:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-33", "mode": "writeback"}]: dispatch 2024-02-14T17:50:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-79590-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:54.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-79590-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:54.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]': finished 2024-02-14T17:50:54.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:54.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:54.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:54.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: osdmap e254: 8 total, 8 up, 8 in 2024-02-14T17:50:54.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:54.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:54.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: pgmap v343: 324 pgs: 6 creating+peering, 26 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:50:54.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-79612-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-79612-50"}]': finished 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]': finished 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-33"}]': finished 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: osdmap e255: 8 total, 8 up, 8 in 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-33", "mode": "writeback"}]: dispatch 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-33", "mode": "writeback"}]: dispatch 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-79590-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-79590-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi100-79590-39"}]': finished 2024-02-14T17:50:54.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2083663302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: osdmap e254: 8 total, 8 up, 8 in 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]: dispatch 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: pgmap v343: 324 pgs: 6 creating+peering, 26 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi100-79612-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi100-79612-50"}]': finished 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi100-79590-39"}]': finished 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-33"}]': finished 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: osdmap e255: 8 total, 8 up, 8 in 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-33", "mode": "writeback"}]: dispatch 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-33", "mode": "writeback"}]: dispatch 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:54.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-79590-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:54.903 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-79590-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:50:55.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:55 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:55.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-33", "mode": "writeback"}]': finished 2024-02-14T17:50:55.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-79590-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:55.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:55 smithi179 ceph-mon[33633]: osdmap e256: 8 total, 8 up, 8 in 2024-02-14T17:50:55.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-79590-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:55.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-79590-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:55.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:55.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:55.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:55.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-33", "mode": "writeback"}]': finished 2024-02-14T17:50:55.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-79590-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:55.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[30138]: osdmap e256: 8 total, 8 up, 8 in 2024-02-14T17:50:55.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-79590-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:55.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-79590-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:55.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:55.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:55.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:50:55.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-33", "mode": "writeback"}]': finished 2024-02-14T17:50:55.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi100-79590-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:50:55.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[39557]: osdmap e256: 8 total, 8 up, 8 in 2024-02-14T17:50:55.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-79590-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:55.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-79590-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:50:55.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:55.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:50:56.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:56 smithi179 ceph-mon[33633]: pgmap v346: 332 pgs: 6 creating+peering, 34 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:56.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:56 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:56.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:56.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:56 smithi100 ceph-mon[39557]: pgmap v346: 332 pgs: 6 creating+peering, 34 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:56.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:56 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:56.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:56.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:56 smithi100 ceph-mon[30138]: pgmap v346: 332 pgs: 6 creating+peering, 34 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:50:56.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:56 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:50:56.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:50:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: osdmap e257: 8 total, 8 up, 8 in 2024-02-14T17:50:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-79590-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-79590-40"}]': finished 2024-02-14T17:50:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33"}]': finished 2024-02-14T17:50:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]': finished 2024-02-14T17:50:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: osdmap e258: 8 total, 8 up, 8 in 2024-02-14T17:50:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: osdmap e257: 8 total, 8 up, 8 in 2024-02-14T17:50:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-79590-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-79590-40"}]': finished 2024-02-14T17:50:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33"}]': finished 2024-02-14T17:50:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]': finished 2024-02-14T17:50:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: osdmap e258: 8 total, 8 up, 8 in 2024-02-14T17:50:57.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:57.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: osdmap e257: 8 total, 8 up, 8 in 2024-02-14T17:50:57.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33"}]: dispatch 2024-02-14T17:50:57.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:50:57.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi100-79590-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi100-79590-40"}]': finished 2024-02-14T17:50:57.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-33"}]': finished 2024-02-14T17:50:57.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi100-79612-50"}]': finished 2024-02-14T17:50:57.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: osdmap e258: 8 total, 8 up, 8 in 2024-02-14T17:50:57.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1805653742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:57.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]: dispatch 2024-02-14T17:50:59.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:58 smithi179 ceph-mon[33633]: pgmap v349: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-14T17:50:59.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:58 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:59.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]': finished 2024-02-14T17:50:59.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:50:58 smithi179 ceph-mon[33633]: osdmap e259: 8 total, 8 up, 8 in 2024-02-14T17:50:59.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:58 smithi100 ceph-mon[39557]: pgmap v349: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-14T17:50:59.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:58 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]': finished 2024-02-14T17:50:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:50:58 smithi100 ceph-mon[39557]: osdmap e259: 8 total, 8 up, 8 in 2024-02-14T17:50:59.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:58 smithi100 ceph-mon[30138]: pgmap v349: 332 pgs: 8 unknown, 324 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-14T17:50:59.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:58 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:50:59.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi100-79612-50"}]': finished 2024-02-14T17:50:59.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:50:58 smithi100 ceph-mon[30138]: osdmap e259: 8 total, 8 up, 8 in 2024-02-14T17:51:00.649 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:51:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[39557]: osdmap e260: 8 total, 8 up, 8 in 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3918806466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[39557]: pgmap v352: 356 pgs: 64 unknown, 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[30138]: osdmap e260: 8 total, 8 up, 8 in 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3918806466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:00.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:00 smithi100 ceph-mon[30138]: pgmap v352: 356 pgs: 64 unknown, 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:51:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:00 smithi179 ceph-mon[33633]: osdmap e260: 8 total, 8 up, 8 in 2024-02-14T17:51:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3918806466' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:00.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:00 smithi179 ceph-mon[33633]: pgmap v352: 356 pgs: 64 unknown, 292 active+clean; 4.6 MiB data, 594 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:51:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]': finished 2024-02-14T17:51:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[30138]: osdmap e261: 8 total, 8 up, 8 in 2024-02-14T17:51:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]': finished 2024-02-14T17:51:01.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[39557]: osdmap e261: 8 total, 8 up, 8 in 2024-02-14T17:51:01.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:01.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:01.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:01.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi100-79612-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi100-79590-40"}]': finished 2024-02-14T17:51:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:01 smithi179 ceph-mon[33633]: osdmap e261: 8 total, 8 up, 8 in 2024-02-14T17:51:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1673042113' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]: dispatch 2024-02-14T17:51:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:02.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:51:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:51:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]': finished 2024-02-14T17:51:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: osdmap e262: 8 total, 8 up, 8 in 2024-02-14T17:51:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-79590-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: pgmap v355: 324 pgs: 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]': finished 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: osdmap e262: 8 total, 8 up, 8 in 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:02.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:02.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:02.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:02.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-79590-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:02.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: pgmap v355: 324 pgs: 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:51:02.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:02 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi100-79590-40"}]': finished 2024-02-14T17:51:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: osdmap e262: 8 total, 8 up, 8 in 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-79590-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: pgmap v355: 324 pgs: 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:51:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:02 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:04.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]': finished 2024-02-14T17:51:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-79590-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[39557]: osdmap e263: 8 total, 8 up, 8 in 2024-02-14T17:51:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-79590-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-35", "mode": "writeback"}]: dispatch 2024-02-14T17:51:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-79612-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-35", "mode": "writeback"}]: dispatch 2024-02-14T17:51:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-79612-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:04.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]': finished 2024-02-14T17:51:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-79590-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[30138]: osdmap e263: 8 total, 8 up, 8 in 2024-02-14T17:51:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-79590-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-35", "mode": "writeback"}]: dispatch 2024-02-14T17:51:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-79612-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-35", "mode": "writeback"}]: dispatch 2024-02-14T17:51:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-79612-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:04.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:03 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:04.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]': finished 2024-02-14T17:51:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi100-79590-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:03 smithi179 ceph-mon[33633]: osdmap e263: 8 total, 8 up, 8 in 2024-02-14T17:51:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-79590-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-35", "mode": "writeback"}]: dispatch 2024-02-14T17:51:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-79612-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-35", "mode": "writeback"}]: dispatch 2024-02-14T17:51:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-79612-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:03 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:05.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:51:05.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-35", "mode": "writeback"}]': finished 2024-02-14T17:51:05.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[30138]: pgmap v358: 324 pgs: 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:05.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[30138]: osdmap e264: 8 total, 8 up, 8 in 2024-02-14T17:51:05.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-79590-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-79590-41"}]': finished 2024-02-14T17:51:05.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-79612-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]': finished 2024-02-14T17:51:05.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[30138]: osdmap e265: 8 total, 8 up, 8 in 2024-02-14T17:51:05.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:51:05.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-35", "mode": "writeback"}]': finished 2024-02-14T17:51:05.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[39557]: pgmap v358: 324 pgs: 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:05.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[39557]: osdmap e264: 8 total, 8 up, 8 in 2024-02-14T17:51:05.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-79590-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-79590-41"}]': finished 2024-02-14T17:51:05.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-79612-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]': finished 2024-02-14T17:51:05.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:04 smithi100 ceph-mon[39557]: osdmap e265: 8 total, 8 up, 8 in 2024-02-14T17:51:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:04 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:51:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-35", "mode": "writeback"}]': finished 2024-02-14T17:51:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:04 smithi179 ceph-mon[33633]: pgmap v358: 324 pgs: 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:04 smithi179 ceph-mon[33633]: osdmap e264: 8 total, 8 up, 8 in 2024-02-14T17:51:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi100-79590-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi100-79590-41"}]': finished 2024-02-14T17:51:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi100-79612-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]': finished 2024-02-14T17:51:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:04 smithi179 ceph-mon[33633]: osdmap e265: 8 total, 8 up, 8 in 2024-02-14T17:51:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:06 smithi100 ceph-mon[30138]: osdmap e266: 8 total, 8 up, 8 in 2024-02-14T17:51:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:06 smithi100 ceph-mon[30138]: pgmap v361: 340 pgs: 16 unknown, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:06 smithi100 ceph-mon[39557]: osdmap e266: 8 total, 8 up, 8 in 2024-02-14T17:51:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:06 smithi100 ceph-mon[39557]: pgmap v361: 340 pgs: 16 unknown, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:06 smithi179 ceph-mon[33633]: osdmap e266: 8 total, 8 up, 8 in 2024-02-14T17:51:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:06 smithi179 ceph-mon[33633]: pgmap v361: 340 pgs: 16 unknown, 324 active+clean; 4.6 MiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:07.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[39557]: osdmap e267: 8 total, 8 up, 8 in 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[30138]: osdmap e267: 8 total, 8 up, 8 in 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:07.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:07.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:07 smithi179 ceph-mon[33633]: osdmap e267: 8 total, 8 up, 8 in 2024-02-14T17:51:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:07.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:07.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:07.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:07.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-79590-41"}]': finished 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]': finished 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]': finished 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: osdmap e268: 8 total, 8 up, 8 in 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: pgmap v364: 324 pgs: 324 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-79590-41"}]': finished 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]': finished 2024-02-14T17:51:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[39557]: osdmap e269: 8 total, 8 up, 8 in 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-79590-41"}]': finished 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]': finished 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]': finished 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: osdmap e268: 8 total, 8 up, 8 in 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: pgmap v364: 324 pgs: 324 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-79590-41"}]': finished 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]': finished 2024-02-14T17:51:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:08 smithi100 ceph-mon[30138]: osdmap e269: 8 total, 8 up, 8 in 2024-02-14T17:51:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi100-79590-41"}]': finished 2024-02-14T17:51:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi100-79612-52"}]': finished 2024-02-14T17:51:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-35"}]': finished 2024-02-14T17:51:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: osdmap e268: 8 total, 8 up, 8 in 2024-02-14T17:51:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-79590-41"}]: dispatch 2024-02-14T17:51:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3297065355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]: dispatch 2024-02-14T17:51:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: pgmap v364: 324 pgs: 324 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:51:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2625072597' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi100-79590-41"}]': finished 2024-02-14T17:51:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi100-79612-52"}]': finished 2024-02-14T17:51:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:08 smithi179 ceph-mon[33633]: osdmap e269: 8 total, 8 up, 8 in 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-79590-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-79590-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-79590-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-79590-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:09.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:09.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:09.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:09.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:09.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-79590-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:09.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-79590-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:09.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:09.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:09.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:10.882 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:51:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-79590-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-79590-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: osdmap e270: 8 total, 8 up, 8 in 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-79612-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-79590-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: pgmap v367: 324 pgs: 324 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:51:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35"}]': finished 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-79590-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-79590-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: osdmap e270: 8 total, 8 up, 8 in 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-79612-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-79590-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: pgmap v367: 324 pgs: 324 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:51:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35"}]': finished 2024-02-14T17:51:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi100-79590-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-79590-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: osdmap e270: 8 total, 8 up, 8 in 2024-02-14T17:51:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-79612-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-79590-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35"}]: dispatch 2024-02-14T17:51:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: pgmap v367: 324 pgs: 324 active+clean; 4.6 MiB data, 579 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:51:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:51:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-35"}]': finished 2024-02-14T17:51:12.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[30138]: osdmap e271: 8 total, 8 up, 8 in 2024-02-14T17:51:12.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:12.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-79612-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-79612-53"}]': finished 2024-02-14T17:51:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-79590-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-79590-42"}]': finished 2024-02-14T17:51:12.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[30138]: osdmap e272: 8 total, 8 up, 8 in 2024-02-14T17:51:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[39557]: osdmap e271: 8 total, 8 up, 8 in 2024-02-14T17:51:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-79612-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-79612-53"}]': finished 2024-02-14T17:51:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-79590-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-79590-42"}]': finished 2024-02-14T17:51:12.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:11 smithi100 ceph-mon[39557]: osdmap e272: 8 total, 8 up, 8 in 2024-02-14T17:51:12.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:51:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:51:12.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:11 smithi179 ceph-mon[33633]: osdmap e271: 8 total, 8 up, 8 in 2024-02-14T17:51:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi100-79612-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi100-79612-53"}]': finished 2024-02-14T17:51:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi100-79590-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi100-79590-42"}]': finished 2024-02-14T17:51:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:11 smithi179 ceph-mon[33633]: osdmap e272: 8 total, 8 up, 8 in 2024-02-14T17:51:13.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:12 smithi100 ceph-mon[39557]: pgmap v370: 308 pgs: 16 unknown, 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:12 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:12 smithi100 ceph-mon[39557]: osdmap e273: 8 total, 8 up, 8 in 2024-02-14T17:51:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:13.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:12 smithi100 ceph-mon[30138]: pgmap v370: 308 pgs: 16 unknown, 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:13.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:12 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:13.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:12 smithi100 ceph-mon[30138]: osdmap e273: 8 total, 8 up, 8 in 2024-02-14T17:51:13.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:13.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:12 smithi179 ceph-mon[33633]: pgmap v370: 308 pgs: 16 unknown, 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:12 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:12 smithi179 ceph-mon[33633]: osdmap e273: 8 total, 8 up, 8 in 2024-02-14T17:51:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:13.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:14.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:14.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:14 smithi179 ceph-mon[33633]: osdmap e274: 8 total, 8 up, 8 in 2024-02-14T17:51:14.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:14.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:14.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:14.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:14.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:14.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:14 smithi179 ceph-mon[33633]: pgmap v373: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[30138]: osdmap e274: 8 total, 8 up, 8 in 2024-02-14T17:51:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[30138]: pgmap v373: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:14.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:14.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[39557]: osdmap e274: 8 total, 8 up, 8 in 2024-02-14T17:51:14.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:14.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:14.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:14.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:14.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:14.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:14 smithi100 ceph-mon[39557]: pgmap v373: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:15.314 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: st api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7021 ms) 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7159 ms) 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6740 ms) 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7022 ms) 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7037 ms) 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7029 ms) 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7044 ms) 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-14T17:51:15.315 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (6880 ms) 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7050 ms) 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (6945 ms) 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7026 ms) 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6988 ms) 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7016 ms) 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7024 ms) 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-14T17:51:15.316 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7029 ms) 2024-02-14T17:51:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-14T17:51:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (6937 ms) 2024-02-14T17:51:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (111948 ms total) 2024-02-14T17:51:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: 2024-02-14T17:51:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [----------] Global test environment tear-down 2024-02-14T17:51:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (209321 ms total) 2024-02-14T17:51:15.317 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio: [ PASSED ] 42 tests. 2024-02-14T17:51:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:15 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]': finished 2024-02-14T17:51:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53"}]': finished 2024-02-14T17:51:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:15 smithi179 ceph-mon[33633]: osdmap e275: 8 total, 8 up, 8 in 2024-02-14T17:51:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]': finished 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53"}]': finished 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[30138]: osdmap e275: 8 total, 8 up, 8 in 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi100-79590-42"}]': finished 2024-02-14T17:51:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi100-79612-53"}]': finished 2024-02-14T17:51:15.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[39557]: osdmap e275: 8 total, 8 up, 8 in 2024-02-14T17:51:15.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:15.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-79612-53"}]: dispatch 2024-02-14T17:51:15.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:15.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1300792845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:15.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]: dispatch 2024-02-14T17:51:16.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-79612-53"}]': finished 2024-02-14T17:51:16.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-37"}]': finished 2024-02-14T17:51:16.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]': finished 2024-02-14T17:51:16.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:16 smithi179 ceph-mon[33633]: osdmap e276: 8 total, 8 up, 8 in 2024-02-14T17:51:16.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-37", "mode": "writeback"}]: dispatch 2024-02-14T17:51:16.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-37", "mode": "writeback"}]: dispatch 2024-02-14T17:51:16.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:16.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:16.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-79612-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:16.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:16 smithi179 ceph-mon[33633]: pgmap v376: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-79612-53"}]': finished 2024-02-14T17:51:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-37"}]': finished 2024-02-14T17:51:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]': finished 2024-02-14T17:51:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[39557]: osdmap e276: 8 total, 8 up, 8 in 2024-02-14T17:51:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-37", "mode": "writeback"}]: dispatch 2024-02-14T17:51:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-37", "mode": "writeback"}]: dispatch 2024-02-14T17:51:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-79612-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[39557]: pgmap v376: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4075901142' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi100-79612-53"}]': finished 2024-02-14T17:51:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-37"}]': finished 2024-02-14T17:51:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi100-79590-42"}]': finished 2024-02-14T17:51:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[30138]: osdmap e276: 8 total, 8 up, 8 in 2024-02-14T17:51:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-37", "mode": "writeback"}]: dispatch 2024-02-14T17:51:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-37", "mode": "writeback"}]: dispatch 2024-02-14T17:51:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-79612-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:16 smithi100 ceph-mon[30138]: pgmap v376: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 584 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:17 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:51:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-37", "mode": "writeback"}]': finished 2024-02-14T17:51:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-79612-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:17 smithi179 ceph-mon[33633]: osdmap e277: 8 total, 8 up, 8 in 2024-02-14T17:51:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-79612-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:51:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-37", "mode": "writeback"}]': finished 2024-02-14T17:51:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-79612-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[39557]: osdmap e277: 8 total, 8 up, 8 in 2024-02-14T17:51:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-79612-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:51:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-37", "mode": "writeback"}]': finished 2024-02-14T17:51:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi100-79612-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:17.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[30138]: osdmap e277: 8 total, 8 up, 8 in 2024-02-14T17:51:17.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-79612-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:17.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:17.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:18 smithi179 ceph-mon[33633]: osdmap e278: 8 total, 8 up, 8 in 2024-02-14T17:51:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:18 smithi179 ceph-mon[33633]: pgmap v379: 324 pgs: 324 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:18 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:51:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-79612-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-79612-54"}]': finished 2024-02-14T17:51:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37"}]': finished 2024-02-14T17:51:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:18 smithi179 ceph-mon[33633]: osdmap e279: 8 total, 8 up, 8 in 2024-02-14T17:51:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[30138]: osdmap e278: 8 total, 8 up, 8 in 2024-02-14T17:51:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[30138]: pgmap v379: 324 pgs: 324 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:51:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-79612-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-79612-54"}]': finished 2024-02-14T17:51:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37"}]': finished 2024-02-14T17:51:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[30138]: osdmap e279: 8 total, 8 up, 8 in 2024-02-14T17:51:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[39557]: osdmap e278: 8 total, 8 up, 8 in 2024-02-14T17:51:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37"}]: dispatch 2024-02-14T17:51:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[39557]: pgmap v379: 324 pgs: 324 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:51:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi100-79612-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi100-79612-54"}]': finished 2024-02-14T17:51:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-37"}]': finished 2024-02-14T17:51:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:18 smithi100 ceph-mon[39557]: osdmap e279: 8 total, 8 up, 8 in 2024-02-14T17:51:19.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:19 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:51:19.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:19 smithi100 ceph-mon[30138]: osdmap e280: 8 total, 8 up, 8 in 2024-02-14T17:51:19.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:19 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:51:19.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:19 smithi100 ceph-mon[39557]: osdmap e280: 8 total, 8 up, 8 in 2024-02-14T17:51:20.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:19 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:51:20.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:19 smithi179 ceph-mon[33633]: osdmap e280: 8 total, 8 up, 8 in 2024-02-14T17:51:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:51:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[39557]: pgmap v382: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:51:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:51:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[39557]: osdmap e281: 8 total, 8 up, 8 in 2024-02-14T17:51:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[30138]: pgmap v382: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:51:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:51:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[30138]: osdmap e281: 8 total, 8 up, 8 in 2024-02-14T17:51:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:20 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:20 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:20 smithi179 ceph-mon[33633]: pgmap v382: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:20 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:51:21.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:20 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:51:21.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:20 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:21.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:20 smithi179 ceph-mon[33633]: osdmap e281: 8 total, 8 up, 8 in 2024-02-14T17:51:21.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:21.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:21.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:21.855 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:21 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:21.855 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:21.855 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-79612-54"}]': finished 2024-02-14T17:51:21.855 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:21 smithi179 ceph-mon[33633]: osdmap e282: 8 total, 8 up, 8 in 2024-02-14T17:51:21.855 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:21.855 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:21.856 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:21.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:21.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-79612-54"}]': finished 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[30138]: osdmap e282: 8 total, 8 up, 8 in 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi100-79612-54"}]': finished 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[39557]: osdmap e282: 8 total, 8 up, 8 in 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:21.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-79612-54"}]: dispatch 2024-02-14T17:51:21.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:22.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:51:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:51:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: pgmap v385: 324 pgs: 4 creating+activating, 28 creating+peering, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:51:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-79612-54"}]': finished 2024-02-14T17:51:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: osdmap e283: 8 total, 8 up, 8 in 2024-02-14T17:51:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-79612-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-79612-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: pgmap v385: 324 pgs: 4 creating+activating, 28 creating+peering, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-79612-54"}]': finished 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: osdmap e283: 8 total, 8 up, 8 in 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:22.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:22.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-79612-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:22.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-79612-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: pgmap v385: 324 pgs: 4 creating+activating, 28 creating+peering, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:51:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2826453867' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi100-79612-54"}]': finished 2024-02-14T17:51:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: osdmap e283: 8 total, 8 up, 8 in 2024-02-14T17:51:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:23.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:23.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-79612-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:23.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-79612-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:24.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-39"}]': finished 2024-02-14T17:51:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-79612-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[30138]: osdmap e284: 8 total, 8 up, 8 in 2024-02-14T17:51:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-39", "mode": "writeback"}]: dispatch 2024-02-14T17:51:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-79612-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-39", "mode": "writeback"}]: dispatch 2024-02-14T17:51:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-79612-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[30138]: pgmap v388: 324 pgs: 4 creating+activating, 28 creating+peering, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:51:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-39"}]': finished 2024-02-14T17:51:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-79612-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[39557]: osdmap e284: 8 total, 8 up, 8 in 2024-02-14T17:51:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-39", "mode": "writeback"}]: dispatch 2024-02-14T17:51:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-79612-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-39", "mode": "writeback"}]: dispatch 2024-02-14T17:51:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-79612-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:24.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:24 smithi100 ceph-mon[39557]: pgmap v388: 324 pgs: 4 creating+activating, 28 creating+peering, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:51:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-39"}]': finished 2024-02-14T17:51:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi100-79612-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:24 smithi179 ceph-mon[33633]: osdmap e284: 8 total, 8 up, 8 in 2024-02-14T17:51:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-39", "mode": "writeback"}]: dispatch 2024-02-14T17:51:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-79612-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-39", "mode": "writeback"}]: dispatch 2024-02-14T17:51:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-79612-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:24 smithi179 ceph-mon[33633]: pgmap v388: 324 pgs: 4 creating+activating, 28 creating+peering, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T17:51:25.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:25 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:51:25.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-39", "mode": "writeback"}]': finished 2024-02-14T17:51:25.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:25 smithi100 ceph-mon[39557]: osdmap e285: 8 total, 8 up, 8 in 2024-02-14T17:51:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:25 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:51:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-39", "mode": "writeback"}]': finished 2024-02-14T17:51:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:25 smithi100 ceph-mon[30138]: osdmap e285: 8 total, 8 up, 8 in 2024-02-14T17:51:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:25.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:25 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:51:25.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-39", "mode": "writeback"}]': finished 2024-02-14T17:51:25.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:25 smithi179 ceph-mon[33633]: osdmap e285: 8 total, 8 up, 8 in 2024-02-14T17:51:25.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:25.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:26.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-79612-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-79612-55"}]': finished 2024-02-14T17:51:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[30138]: osdmap e286: 8 total, 8 up, 8 in 2024-02-14T17:51:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[30138]: pgmap v391: 332 pgs: 8 unknown, 4 creating+activating, 28 creating+peering, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:26.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-79612-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-79612-55"}]': finished 2024-02-14T17:51:26.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:26.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[39557]: osdmap e286: 8 total, 8 up, 8 in 2024-02-14T17:51:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[39557]: pgmap v391: 332 pgs: 8 unknown, 4 creating+activating, 28 creating+peering, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:26.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:26 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi100-79612-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi100-79612-55"}]': finished 2024-02-14T17:51:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:51:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:26 smithi179 ceph-mon[33633]: osdmap e286: 8 total, 8 up, 8 in 2024-02-14T17:51:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39"}]: dispatch 2024-02-14T17:51:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:26 smithi179 ceph-mon[33633]: pgmap v391: 332 pgs: 8 unknown, 4 creating+activating, 28 creating+peering, 292 active+clean; 4.6 MiB data, 589 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:26 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:26.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:26 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-02-14T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-14T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-14T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: seed 79838 2024-02-14T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-14T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (314 ms) 2024-02-14T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-14T17:51:27.355 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (15828 ms) 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (123 ms) 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (16265 ms total) 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (6929 ms) 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8015 ms) 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9951 ms) 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: my_snaps [3] 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: my_snaps [4,3] 2024-02-14T17:51:27.356 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: my_snaps [5,4,3] 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: promoting some heads 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: promoting from clones for snap 3 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: waiting for scrubs... 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: done waiting 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46101 ms) 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9921 ms) 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7988 ms) 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-14T17:51:27.357 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8271 ms) 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (7803 ms) 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10034 ms) 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8907 ms) 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10074 ms) 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12194 ms) 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7738 ms) 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-14T17:51:27.358 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8021 ms) 2024-02-14T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-14T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13074 ms) 2024-02-14T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-14T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7925 ms) 2024-02-14T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-14T17:51:27.359 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8031 ms) 2024-02-14T17:51:27.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:27 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:27.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:27 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:51:27.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39"}]': finished 2024-02-14T17:51:27.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:27 smithi100 ceph-mon[39557]: osdmap e287: 8 total, 8 up, 8 in 2024-02-14T17:51:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:27 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:27 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:51:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39"}]': finished 2024-02-14T17:51:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:27 smithi100 ceph-mon[30138]: osdmap e287: 8 total, 8 up, 8 in 2024-02-14T17:51:27.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:27 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:27 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:51:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-39"}]': finished 2024-02-14T17:51:27.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:27 smithi179 ceph-mon[33633]: osdmap e287: 8 total, 8 up, 8 in 2024-02-14T17:51:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:28 smithi100 ceph-mon[39557]: osdmap e288: 8 total, 8 up, 8 in 2024-02-14T17:51:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:28 smithi100 ceph-mon[39557]: pgmap v394: 292 pgs: 292 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-14T17:51:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:28 smithi100 ceph-mon[30138]: osdmap e288: 8 total, 8 up, 8 in 2024-02-14T17:51:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:28 smithi100 ceph-mon[30138]: pgmap v394: 292 pgs: 292 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-14T17:51:28.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:28 smithi179 ceph-mon[33633]: osdmap e288: 8 total, 8 up, 8 in 2024-02-14T17:51:28.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:28.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:28.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:28 smithi179 ceph-mon[33633]: pgmap v394: 292 pgs: 292 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-14T17:51:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]': finished 2024-02-14T17:51:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[30138]: osdmap e289: 8 total, 8 up, 8 in 2024-02-14T17:51:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]': finished 2024-02-14T17:51:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[39557]: osdmap e289: 8 total, 8 up, 8 in 2024-02-14T17:51:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:29.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:29.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:29.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi100-79612-55"}]': finished 2024-02-14T17:51:29.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:29 smithi179 ceph-mon[33633]: osdmap e289: 8 total, 8 up, 8 in 2024-02-14T17:51:29.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3229071198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:29.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]: dispatch 2024-02-14T17:51:29.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:29.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:30.649 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:51:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]': finished 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: osdmap e290: 8 total, 8 up, 8 in 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-79612-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-79612-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[39557]: pgmap v397: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]': finished 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: osdmap e290: 8 total, 8 up, 8 in 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-79612-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-79612-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:30 smithi100 ceph-mon[30138]: pgmap v397: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-14T17:51:30.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi100-79612-55"}]': finished 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: osdmap e290: 8 total, 8 up, 8 in 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-79612-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-79612-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:30.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:30 smithi179 ceph-mon[33633]: pgmap v397: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-14T17:51:31.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:31.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-79612-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:31.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[30138]: osdmap e291: 8 total, 8 up, 8 in 2024-02-14T17:51:31.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:51:31.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-79612-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:31.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:51:31.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-79612-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:31.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:31.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-79612-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:31.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[39557]: osdmap e291: 8 total, 8 up, 8 in 2024-02-14T17:51:31.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:51:31.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-79612-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:31.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:51:31.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-79612-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:32.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:51:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:51:32.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi100-79612-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:31 smithi179 ceph-mon[33633]: osdmap e291: 8 total, 8 up, 8 in 2024-02-14T17:51:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:51:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-79612-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:51:32.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-79612-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSearted 0x5648c4772580 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47720e0 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47721f0 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4772320 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4772690 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4772b20 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4772c30 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4773200 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47727c0 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4772430 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47728f0 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4772a00 2024-02-14T17:51:32.465 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4772d60 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4772e70 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4772f80 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47730d0 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4773810 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4773370 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47734a0 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47735b0 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4773700 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4773e00 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4773960 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4773a70 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4773bc0 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4773cd0 2024-02-14T17:51:32.466 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47743f0 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4773f50 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4774060 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47741c0 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4774500 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4774970 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4774ae0 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47742d0 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4774610 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4774760 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4774bf0 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4774d00 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4774e50 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4775110 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4775580 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4775690 2024-02-14T17:51:32.467 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4775c60 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47757c0 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4774fe0 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47758f0 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4775a00 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4775b10 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4775d70 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4776250 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4776360 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47688b0 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47689c0 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4776950 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47764b0 2024-02-14T17:51:32.468 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47765e0 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4776710 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4776840 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4776f00 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4776a60 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4776b90 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4776ca0 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4776dd0 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47774d0 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4777030 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4777180 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47772b0 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47775e0 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4777a70 2024-02-14T17:51:32.469 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4777ba0 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47776f0 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4777800 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4777950 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4777cf0 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4777e80 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4778470 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4777fd0 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4775ea0 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4778d80 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47788e0 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4775fd0 2024-02-14T17:51:32.470 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47760e0 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: osdmap e292: 8 total, 8 up, 8 in 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: pgmap v400: 324 pgs: 324 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-79612-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-79612-56"}]': finished 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: osdmap e293: 8 total, 8 up, 8 in 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:51:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: osdmap e292: 8 total, 8 up, 8 in 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: pgmap v400: 324 pgs: 324 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-79612-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-79612-56"}]': finished 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: osdmap e293: 8 total, 8 up, 8 in 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:51:32.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:51:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:51:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: osdmap e292: 8 total, 8 up, 8 in 2024-02-14T17:51:33.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:33.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: pgmap v400: 324 pgs: 324 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:33.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:33.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi100-79612-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi100-79612-56"}]': finished 2024-02-14T17:51:33.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:51:33.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: osdmap e293: 8 total, 8 up, 8 in 2024-02-14T17:51:33.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:51:33.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:51:33.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:33 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:33.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:51:33.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:33 smithi100 ceph-mon[30138]: osdmap e294: 8 total, 8 up, 8 in 2024-02-14T17:51:33.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:33 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:33.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:51:33.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:33 smithi100 ceph-mon[39557]: osdmap e294: 8 total, 8 up, 8 in 2024-02-14T17:51:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:33 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:51:34.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:33 smithi179 ceph-mon[33633]: osdmap e294: 8 total, 8 up, 8 in 2024-02-14T17:51:34.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[39557]: pgmap v403: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:34.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[39557]: osdmap e295: 8 total, 8 up, 8 in 2024-02-14T17:51:34.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:34.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:34.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:34.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:34.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41"}]: dispatch 2024-02-14T17:51:34.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41"}]: dispatch 2024-02-14T17:51:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[30138]: pgmap v403: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[30138]: osdmap e295: 8 total, 8 up, 8 in 2024-02-14T17:51:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41"}]: dispatch 2024-02-14T17:51:34.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41"}]: dispatch 2024-02-14T17:51:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:34 smithi179 ceph-mon[33633]: pgmap v403: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:34 smithi179 ceph-mon[33633]: osdmap e295: 8 total, 8 up, 8 in 2024-02-14T17:51:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:35.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:35.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:35.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:35.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41"}]: dispatch 2024-02-14T17:51:35.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41"}]: dispatch 2024-02-14T17:51:36.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]': finished 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41"}]': finished 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[30138]: osdmap e296: 8 total, 8 up, 8 in 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[30138]: pgmap v406: 324 pgs: 324 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]': finished 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41"}]': finished 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[39557]: osdmap e296: 8 total, 8 up, 8 in 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:36 smithi100 ceph-mon[39557]: pgmap v406: 324 pgs: 324 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi100-79612-56"}]': finished 2024-02-14T17:51:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-41"}]': finished 2024-02-14T17:51:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:36 smithi179 ceph-mon[33633]: osdmap e296: 8 total, 8 up, 8 in 2024-02-14T17:51:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/237085632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]: dispatch 2024-02-14T17:51:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:36 smithi179 ceph-mon[33633]: pgmap v406: 324 pgs: 324 active+clean; 8.4 MiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:37.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]': finished 2024-02-14T17:51:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[30138]: osdmap e297: 8 total, 8 up, 8 in 2024-02-14T17:51:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-79612-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-79612-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:37.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]': finished 2024-02-14T17:51:37.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[39557]: osdmap e297: 8 total, 8 up, 8 in 2024-02-14T17:51:37.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-79612-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-79612-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:37.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi100-79612-56"}]': finished 2024-02-14T17:51:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:37 smithi179 ceph-mon[33633]: osdmap e297: 8 total, 8 up, 8 in 2024-02-14T17:51:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-79612-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:37.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-79612-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-79612-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:38.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:38 smithi179 ceph-mon[33633]: osdmap e298: 8 total, 8 up, 8 in 2024-02-14T17:51:38.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-79612-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:38.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-79612-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:38.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:38.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:38.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:38 smithi179 ceph-mon[33633]: pgmap v409: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:38.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-79612-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[39557]: osdmap e298: 8 total, 8 up, 8 in 2024-02-14T17:51:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-79612-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-79612-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[39557]: pgmap v409: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:38.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi100-79612-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:38.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[30138]: osdmap e298: 8 total, 8 up, 8 in 2024-02-14T17:51:38.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-79612-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:38.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-79612-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:38.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:38.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:38.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:38 smithi100 ceph-mon[30138]: pgmap v409: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:39.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:39 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:39.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:39.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-79838-6","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:51:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:39 smithi179 ceph-mon[33633]: osdmap e299: 8 total, 8 up, 8 in 2024-02-14T17:51:39.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:39.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-79838-6","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[30138]: osdmap e299: 8 total, 8 up, 8 in 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-79838-6","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[39557]: osdmap e299: 8 total, 8 up, 8 in 2024-02-14T17:51:39.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:40.801 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:51:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:40.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-79612-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-79612-57"}]': finished 2024-02-14T17:51:40.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:40.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[39557]: osdmap e300: 8 total, 8 up, 8 in 2024-02-14T17:51:40.802 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:51:40.802 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:51:40.802 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[39557]: pgmap v412: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-79612-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-79612-57"}]': finished 2024-02-14T17:51:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[30138]: osdmap e300: 8 total, 8 up, 8 in 2024-02-14T17:51:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:51:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:51:40.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:40 smithi100 ceph-mon[30138]: pgmap v412: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi100-79612-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi100-79612-57"}]': finished 2024-02-14T17:51:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:40 smithi179 ceph-mon[33633]: osdmap e300: 8 total, 8 up, 8 in 2024-02-14T17:51:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:51:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-14T17:51:40.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:40 smithi179 ceph-mon[33633]: pgmap v412: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:41.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:41 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_count","val": "8"}]': finished 2024-02-14T17:51:41.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:41 smithi179 ceph-mon[33633]: osdmap e301: 8 total, 8 up, 8 in 2024-02-14T17:51:41.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:41 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:41.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:41 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:41.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:41.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:41 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:41.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_count","val": "8"}]': finished 2024-02-14T17:51:41.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[30138]: osdmap e301: 8 total, 8 up, 8 in 2024-02-14T17:51:41.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:41.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:41.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:41.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:41.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_count","val": "8"}]': finished 2024-02-14T17:51:41.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[39557]: osdmap e301: 8 total, 8 up, 8 in 2024-02-14T17:51:41.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:41.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:51:41.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:51:41.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:41 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:42.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:51:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:51:42.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_misctRead 2024-02-14T17:51:42.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-14T17:51:42.667 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: ok, hit_set contains 273:602f83fe:::foo:head 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9041 ms) 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg_num = 32 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 0 ls 1707933103,0 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 1 ls 1707933103,0 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 2 ls 1707933103,0 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 3 ls 1707933103,0 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 4 ls 1707933103,0 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 5 ls 1707933103,0 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 6 ls 1707933103,0 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 7 ls 1707933103,0 2024-02-14T17:51:42.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 8 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 9 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 10 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 11 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 12 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 13 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 14 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 15 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 16 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 17 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 18 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 19 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 20 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 21 ls 1707933103,0 2024-02-14T17:51:42.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 22 ls 1707933103,0 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 23 ls 1707933103,0 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 24 ls 1707933103,0 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 25 ls 1707933103,0 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 26 ls 1707933103,0 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 27 ls 1707933103,0 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 28 ls 1707933103,0 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 29 ls 1707933103,0 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 30 ls 1707933103,0 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg 31 ls 1707933103,0 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: pg_num = 32 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:6cac518f:::0:head 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:02547ec2:::1:head 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:f905c69b:::2:head 2024-02-14T17:51:42.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:cfc208b3:::3:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d83876eb:::4:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:b29083e3:::5:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c4fdafeb:::6:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:5c6b0b28:::7:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:bd63b0f1:::8:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:e960b815:::9:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:52ea6a34:::10:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:89d3ae78:::11:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:de5d7c5f:::12:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:566253c9:::13:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:62a1935d:::14:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:863748b0:::15:head 2024-02-14T17:51:42.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:3958e169:::16:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:4d4dabf9:::17:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:8391935d:::18:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:28883081:::19:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:69259c59:::20:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:4bdb80b7:::21:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:a11c5d71:::22:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:271af37b:::23:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:95b121be:::24:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:58d1031b:::25:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:0a050783:::26:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c709704c:::27:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:cbe56eaf:::28:head 2024-02-14T17:51:42.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:86b4b162:::29:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:70d89383:::30:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:dd450c7c:::31:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:6d5729b1:::32:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c388f3fb:::33:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:56cfea31:::34:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:9dbc1bf7:::35:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:40b74ccd:::36:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:4d5aaf42:::37:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:920f362c:::38:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:6cc53222:::39:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:9cad833f:::40:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:1ea84d41:::41:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c4480ef6:::42:head 2024-02-14T17:51:42.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:a694361e:::43:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d1bd33e9:::44:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:ddc2cd5d:::45:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:2b782207:::46:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:7b187fca:::47:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:90ecdf6f:::48:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:a5ed95fe:::49:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:ea0eaa55:::50:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:f33ef17b:::51:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:a0d1b2f6:::52:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:60c5229e:::53:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:edcbc575:::54:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:102cf253:::55:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:efb7fb0b:::56:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:50d0a326:::57:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d4dc5daf:::58:head 2024-02-14T17:51:42.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:3a130462:::59:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:ec87ed71:::60:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d5bc9454:::61:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:3ddfe313:::62:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:7c2816b9:::63:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:47e00e4d:::64:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c6410c18:::65:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:b48ed237:::66:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:cd63ad31:::67:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:b179e92b:::68:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:0d9f741a:::69:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:6d3352ae:::70:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c6d5c19e:::71:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:bc4729c3:::72:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:77e930b9:::73:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:0abeecfd:::74:head 2024-02-14T17:51:42.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:b7c37e15:::75:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:b6378398:::76:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:02bd68de:::77:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:cc795d2d:::78:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:630d4fea:::79:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:e0d29ef5:::80:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:fd6f13d2:::81:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:606461d5:::82:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:eadbdc43:::83:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:8761d0bb:::84:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:9ef0186f:::85:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:e0d41294:::86:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:961de695:::87:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:1423148f:::88:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:633a8fa2:::89:head 2024-02-14T17:51:42.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:a8653809:::90:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:3dac8b33:::91:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:35aad435:::92:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:f6dcc343:::93:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:dbbdad87:::94:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:1cb48ce0:::95:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:03cd461c:::96:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:17a4ea99:::97:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:9993c9a7:::98:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:6394211c:::99:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:94c7ae57:::100:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:6fdee5bb:::101:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:9a477fd1:::102:head 2024-02-14T17:51:42.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:eb850916:::103:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:affc56b9:::104:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:b42dc814:::105:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:f319f8f0:::106:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:9a40b9de:::107:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:8b524f28:::108:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:e3de589f:::109:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:90f90a5b:::110:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:a7b4f1d7:::111:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:af51766e:::112:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:b6f90bd1:::113:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:e0261208:::114:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c9569ef7:::115:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:61bebe50:::116:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:fe93412b:::117:head 2024-02-14T17:51:42.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d3d38bee:::118:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:3100ba0c:::119:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d0560ada:::120:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:f0ea8b35:::121:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:766f231a:::122:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:a07a2582:::123:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:bd7c6b3a:::124:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:fb2ddaff:::125:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:4408e1fe:::126:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:ee1df7a7:::127:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c3002909:::128:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:4f48ffa9:::129:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:edf38733:::130:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c08425c0:::131:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:5f902d98:::132:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:41ea2c93:::133:head 2024-02-14T17:51:42.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:813cee13:::134:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:0131818d:::135:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:26ba5a85:::136:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:381b8a5a:::137:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:28797e47:::138:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:bfca7f22:::139:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:36807075:::140:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:80b03975:::141:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:5c15709b:::142:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:f39ea15e:::143:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:ea992956:::144:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:48887b1c:::145:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:9f24a9dd:::146:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:987f100b:::147:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d2dd3581:::148:head 2024-02-14T17:51:42.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:7fed1808:::149:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c80b70e9:::150:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:85ed90f9:::151:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:36428b24:::152:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d044c34a:::153:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:7c18bf58:::154:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d1c21232:::155:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:a7a3c575:::156:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:87da0633:::157:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d5ac3822:::158:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:3f20522d:::159:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:6ca26563:::160:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:532ce135:::161:head 2024-02-14T17:51:42.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:c78863e6:::162:head 2024-02-14T17:51:42.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:51:42.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: osdmap e302: 8 total, 8 up, 8 in 2024-02-14T17:51:42.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:51:42.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:51:42.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:42.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:42.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: pgmap v415: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:42.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:42.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-14T17:51:42.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]': finished 2024-02-14T17:51:42.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:42 smithi179 ceph-mon[33633]: osdmap e303: 8 total, 8 up, 8 in 2024-02-14T17:51:42.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:51:42.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: osdmap e302: 8 total, 8 up, 8 in 2024-02-14T17:51:42.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:51:42.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:51:42.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:42.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:42.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: pgmap v415: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:42.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:42.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-14T17:51:42.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]': finished 2024-02-14T17:51:42.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[30138]: osdmap e303: 8 total, 8 up, 8 in 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: osdmap e302: 8 total, 8 up, 8 in 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: pgmap v415: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi100-79612-57"}]': finished 2024-02-14T17:51:42.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:42 smithi100 ceph-mon[39557]: osdmap e303: 8 total, 8 up, 8 in 2024-02-14T17:51:43.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:43.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:43.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-79838-43","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:51:43.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:43.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:43.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43"}]: dispatch 2024-02-14T17:51:43.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43"}]: dispatch 2024-02-14T17:51:43.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]': finished 2024-02-14T17:51:43.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43"}]': finished 2024-02-14T17:51:43.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:43 smithi179 ceph-mon[33633]: osdmap e304: 8 total, 8 up, 8 in 2024-02-14T17:51:43.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:43.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:43.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-79838-43","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:51:43.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:43.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:43.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43"}]: dispatch 2024-02-14T17:51:43.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43"}]: dispatch 2024-02-14T17:51:43.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]': finished 2024-02-14T17:51:43.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43"}]': finished 2024-02-14T17:51:43.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[39557]: osdmap e304: 8 total, 8 up, 8 in 2024-02-14T17:51:43.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4247733673' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:43.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]: dispatch 2024-02-14T17:51:43.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi100-79838-43","var": "pg_num","format": "json"}]: dispatch 2024-02-14T17:51:43.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:43.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:51:43.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43"}]: dispatch 2024-02-14T17:51:43.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43"}]: dispatch 2024-02-14T17:51:43.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi100-79612-57"}]': finished 2024-02-14T17:51:43.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-43"}]': finished 2024-02-14T17:51:43.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:43 smithi100 ceph-mon[30138]: osdmap e304: 8 total, 8 up, 8 in 2024-02-14T17:51:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-79612-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:44.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-79612-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:44.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:44 smithi179 ceph-mon[33633]: pgmap v418: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:44.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-79612-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-79612-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[30138]: pgmap v418: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:44.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-79612-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:44.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-79612-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:44.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:44 smithi100 ceph-mon[39557]: pgmap v418: 324 pgs: 324 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:46.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-79612-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:46.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:45 smithi179 ceph-mon[33633]: osdmap e305: 8 total, 8 up, 8 in 2024-02-14T17:51:46.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-79612-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:46.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-79612-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:46.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-79612-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:46.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:45 smithi100 ceph-mon[30138]: osdmap e305: 8 total, 8 up, 8 in 2024-02-14T17:51:46.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-79612-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:46.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-79612-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:46.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi100-79612-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:46.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:45 smithi100 ceph-mon[39557]: osdmap e305: 8 total, 8 up, 8 in 2024-02-14T17:51:46.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-79612-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:46.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-79612-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:47.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:46 smithi179 ceph-mon[33633]: pgmap v421: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:47.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:46 smithi179 ceph-mon[33633]: osdmap e306: 8 total, 8 up, 8 in 2024-02-14T17:51:47.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:47.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:47.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:46 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:47.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-79612-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-79612-58"}]': finished 2024-02-14T17:51:47.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:47.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:46 smithi179 ceph-mon[33633]: osdmap e307: 8 total, 8 up, 8 in 2024-02-14T17:51:47.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:47.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[30138]: pgmap v421: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[30138]: osdmap e306: 8 total, 8 up, 8 in 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-79612-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-79612-58"}]': finished 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[30138]: osdmap e307: 8 total, 8 up, 8 in 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:47.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[39557]: pgmap v421: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:47.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[39557]: osdmap e306: 8 total, 8 up, 8 in 2024-02-14T17:51:47.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:47.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:47.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:47.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi100-79612-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi100-79612-58"}]': finished 2024-02-14T17:51:47.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:51:47.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[39557]: osdmap e307: 8 total, 8 up, 8 in 2024-02-14T17:51:47.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:47.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:51:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:48 smithi179 ceph-mon[33633]: osdmap e308: 8 total, 8 up, 8 in 2024-02-14T17:51:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:51:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:51:48.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:48 smithi179 ceph-mon[33633]: pgmap v424: 332 pgs: 8 unknown, 32 creating+peering, 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:48.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:48 smithi100 ceph-mon[30138]: osdmap e308: 8 total, 8 up, 8 in 2024-02-14T17:51:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:51:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:51:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:48 smithi100 ceph-mon[30138]: pgmap v424: 332 pgs: 8 unknown, 32 creating+peering, 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:48.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:51:48.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:48 smithi100 ceph-mon[39557]: osdmap e308: 8 total, 8 up, 8 in 2024-02-14T17:51:48.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:51:48.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T17:51:48.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:48 smithi100 ceph-mon[39557]: pgmap v424: 332 pgs: 8 unknown, 32 creating+peering, 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:49.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:49 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_count","val": "3"}]': finished 2024-02-14T17:51:49.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:49 smithi179 ceph-mon[33633]: osdmap e309: 8 total, 8 up, 8 in 2024-02-14T17:51:49.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:51:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:49 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:51:49.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:49 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:49.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_count","val": "3"}]': finished 2024-02-14T17:51:49.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[30138]: osdmap e309: 8 total, 8 up, 8 in 2024-02-14T17:51:49.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:51:49.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:49.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:51:49.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_count","val": "3"}]': finished 2024-02-14T17:51:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[39557]: osdmap e309: 8 total, 8 up, 8 in 2024-02-14T17:51:49.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:51:49.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:49.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T17:51:49.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:49 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:50.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_period","val": "3"}]': finished 2024-02-14T17:51:50.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]': finished 2024-02-14T17:51:50.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:50 smithi179 ceph-mon[33633]: osdmap e310: 8 total, 8 up, 8 in 2024-02-14T17:51:50.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:51:50.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:50.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:51:50.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:50.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:50 smithi179 ceph-mon[33633]: pgmap v427: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:50.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:51:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:51:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:51:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_period","val": "3"}]': finished 2024-02-14T17:51:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]': finished 2024-02-14T17:51:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[30138]: osdmap e310: 8 total, 8 up, 8 in 2024-02-14T17:51:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:51:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:51:50.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[30138]: pgmap v427: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_period","val": "3"}]': finished 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi100-79612-58"}]': finished 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[39557]: osdmap e310: 8 total, 8 up, 8 in 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3223623133' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]: dispatch 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[39557]: pgmap v427: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:50.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:51:51.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]': finished 2024-02-14T17:51:51.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: osdmap e311: 8 total, 8 up, 8 in 2024-02-14T17:51:51.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:51:51.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:51:51.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:51.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:51.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:51.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T17:51:51.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:51.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: osdmap e312: 8 total, 8 up, 8 in 2024-02-14T17:51:51.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-79612-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:51.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]': finished 2024-02-14T17:51:51.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: osdmap e311: 8 total, 8 up, 8 in 2024-02-14T17:51:51.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:51:51.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:51:51.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:51.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:51.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:51.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T17:51:51.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:51.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: osdmap e312: 8 total, 8 up, 8 in 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-79612-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi100-79612-58"}]': finished 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: osdmap e311: 8 total, 8 up, 8 in 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: osdmap e312: 8 total, 8 up, 8 in 2024-02-14T17:51:51.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-79612-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:52.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:51:51 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:51:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:52 smithi179 ceph-mon[33633]: pgmap v430: 324 pgs: 324 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:52 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:53.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:52 smithi100 ceph-mon[39557]: pgmap v430: 324 pgs: 324 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:53.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:52 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:53.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:52 smithi100 ceph-mon[30138]: pgmap v430: 324 pgs: 324 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:53.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:52 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:51:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:53 smithi179 ceph-mon[33633]: osdmap e313: 8 total, 8 up, 8 in 2024-02-14T17:51:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:53 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-79612-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]': finished 2024-02-14T17:51:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:53 smithi179 ceph-mon[33633]: osdmap e314: 8 total, 8 up, 8 in 2024-02-14T17:51:54.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:53 smithi100 ceph-mon[39557]: osdmap e313: 8 total, 8 up, 8 in 2024-02-14T17:51:54.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:53 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:54.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-79612-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]': finished 2024-02-14T17:51:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:53 smithi100 ceph-mon[39557]: osdmap e314: 8 total, 8 up, 8 in 2024-02-14T17:51:54.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:53 smithi100 ceph-mon[30138]: osdmap e313: 8 total, 8 up, 8 in 2024-02-14T17:51:54.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:53 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:51:54.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi100-79612-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]': finished 2024-02-14T17:51:54.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:53 smithi100 ceph-mon[30138]: osdmap e314: 8 total, 8 up, 8 in 2024-02-14T17:51:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:54 smithi179 ceph-mon[33633]: pgmap v433: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:54 smithi179 ceph-mon[33633]: osdmap e315: 8 total, 8 up, 8 in 2024-02-14T17:51:55.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:54 smithi100 ceph-mon[30138]: pgmap v433: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:55.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:54 smithi100 ceph-mon[30138]: osdmap e315: 8 total, 8 up, 8 in 2024-02-14T17:51:55.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:54 smithi100 ceph-mon[39557]: pgmap v433: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:51:55.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:54 smithi100 ceph-mon[39557]: osdmap e315: 8 total, 8 up, 8 in 2024-02-14T17:51:56.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:56 smithi179 ceph-mon[33633]: osdmap e316: 8 total, 8 up, 8 in 2024-02-14T17:51:56.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:56.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:56 smithi179 ceph-mon[33633]: pgmap v436: 324 pgs: 324 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:56.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:56 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:56.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:56 smithi100 ceph-mon[30138]: osdmap e316: 8 total, 8 up, 8 in 2024-02-14T17:51:56.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:56.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:56 smithi100 ceph-mon[30138]: pgmap v436: 324 pgs: 324 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:56.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:56 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:56.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:56 smithi100 ceph-mon[39557]: osdmap e316: 8 total, 8 up, 8 in 2024-02-14T17:51:56.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:56.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:56 smithi100 ceph-mon[39557]: pgmap v436: 324 pgs: 324 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:51:56.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:56 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:51:57.508 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:7462ddf6:::o.RoundTripAppendPP (2894 ms) 2024-02-14T17:51:57.508 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3017 ms) 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3011 ms) 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (2971 ms) 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: setting pool EIO 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: max_success 99, min_failed 100 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4015 ms) 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3010 ms) 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (124311 ms total) 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: 2024-02-14T17:51:57.509 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3251 ms) 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (8814 ms) 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14920 ms) 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3010 ms) 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29995 ms total) 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3014 ms) 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3014 ms total) 2024-02-14T17:51:57.510 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13997 ms) 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6982 ms) 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7015 ms) 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7020 ms) 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3017 ms) 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7029 ms) 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-14T17:51:57.511 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6938 ms) 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7018 ms) 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7038 ms) 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7031 ms) 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7039 ms) 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7047 ms) 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-14T17:51:57.512 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7024 ms) 2024-02-14T17:51:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]': finished 2024-02-14T17:51:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:57 smithi179 ceph-mon[33633]: osdmap e317: 8 total, 8 up, 8 in 2024-02-14T17:51:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:57.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]': finished 2024-02-14T17:51:57.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:57 smithi100 ceph-mon[30138]: osdmap e317: 8 total, 8 up, 8 in 2024-02-14T17:51:57.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi100-79612-59"}]': finished 2024-02-14T17:51:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:57 smithi100 ceph-mon[39557]: osdmap e317: 8 total, 8 up, 8 in 2024-02-14T17:51:57.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-79612-59"}]: dispatch 2024-02-14T17:51:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-79612-59"}]': finished 2024-02-14T17:51:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:58 smithi179 ceph-mon[33633]: osdmap e318: 8 total, 8 up, 8 in 2024-02-14T17:51:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:58 smithi179 ceph-mon[33633]: pgmap v439: 324 pgs: 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:51:58.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-79612-59"}]': finished 2024-02-14T17:51:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:58 smithi100 ceph-mon[30138]: osdmap e318: 8 total, 8 up, 8 in 2024-02-14T17:51:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:58 smithi100 ceph-mon[30138]: pgmap v439: 324 pgs: 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:51:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3848657932' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi100-79612-59"}]': finished 2024-02-14T17:51:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:58 smithi100 ceph-mon[39557]: osdmap e318: 8 total, 8 up, 8 in 2024-02-14T17:51:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:58 smithi100 ceph-mon[39557]: pgmap v439: 324 pgs: 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:51:59.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:59 smithi179 ceph-mon[33633]: osdmap e319: 8 total, 8 up, 8 in 2024-02-14T17:51:59.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:51:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1327919371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:59.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:59 smithi100 ceph-mon[30138]: osdmap e319: 8 total, 8 up, 8 in 2024-02-14T17:51:59.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:51:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1327919371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:51:59.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:59 smithi100 ceph-mon[39557]: osdmap e319: 8 total, 8 up, 8 in 2024-02-14T17:51:59.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:51:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1327919371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1327919371' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:00 smithi179 ceph-mon[33633]: osdmap e320: 8 total, 8 up, 8 in 2024-02-14T17:52:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:00 smithi179 ceph-mon[33633]: pgmap v442: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:52:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:52:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:52:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1327919371' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:00 smithi100 ceph-mon[30138]: osdmap e320: 8 total, 8 up, 8 in 2024-02-14T17:52:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:00 smithi100 ceph-mon[30138]: pgmap v442: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:52:00.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1327919371' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi100-79612-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:00.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:00 smithi100 ceph-mon[39557]: osdmap e320: 8 total, 8 up, 8 in 2024-02-14T17:52:00.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:00 smithi100 ceph-mon[39557]: pgmap v442: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:52:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:01 smithi179 ceph-mon[33633]: osdmap e321: 8 total, 8 up, 8 in 2024-02-14T17:52:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-79612-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:01.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-79612-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:01.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[30138]: osdmap e321: 8 total, 8 up, 8 in 2024-02-14T17:52:01.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-79612-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:01.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-79612-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[39557]: osdmap e321: 8 total, 8 up, 8 in 2024-02-14T17:52:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-79612-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:01.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-79612-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:02.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:52:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:52:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-79612-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:02 smithi179 ceph-mon[33633]: osdmap e322: 8 total, 8 up, 8 in 2024-02-14T17:52:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-79612-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-79612-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:02 smithi179 ceph-mon[33633]: pgmap v445: 324 pgs: 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:52:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:02 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-79612-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:02.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[39557]: osdmap e322: 8 total, 8 up, 8 in 2024-02-14T17:52:02.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-79612-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:02.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-79612-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:02.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[39557]: pgmap v445: 324 pgs: 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:52:02.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:02.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi100-79612-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:02.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[30138]: osdmap e322: 8 total, 8 up, 8 in 2024-02-14T17:52:02.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-79612-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:02.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-79612-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:02.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[30138]: pgmap v445: 324 pgs: 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:52:02.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:02 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:03 smithi179 ceph-mon[33633]: osdmap e323: 8 total, 8 up, 8 in 2024-02-14T17:52:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45"}]: dispatch 2024-02-14T17:52:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45"}]: dispatch 2024-02-14T17:52:03.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:03 smithi100 ceph-mon[30138]: osdmap e323: 8 total, 8 up, 8 in 2024-02-14T17:52:03.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:03.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45"}]: dispatch 2024-02-14T17:52:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45"}]: dispatch 2024-02-14T17:52:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:03 smithi100 ceph-mon[39557]: osdmap e323: 8 total, 8 up, 8 in 2024-02-14T17:52:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45"}]: dispatch 2024-02-14T17:52:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45"}]: dispatch 2024-02-14T17:52:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-79612-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-79612-61"}]': finished 2024-02-14T17:52:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45"}]': finished 2024-02-14T17:52:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:04 smithi179 ceph-mon[33633]: osdmap e324: 8 total, 8 up, 8 in 2024-02-14T17:52:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:04 smithi179 ceph-mon[33633]: pgmap v448: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:52:04.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-79612-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-79612-61"}]': finished 2024-02-14T17:52:04.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45"}]': finished 2024-02-14T17:52:04.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:04 smithi100 ceph-mon[30138]: osdmap e324: 8 total, 8 up, 8 in 2024-02-14T17:52:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:04 smithi100 ceph-mon[30138]: pgmap v448: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:52:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi100-79612-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi100-79612-61"}]': finished 2024-02-14T17:52:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-45"}]': finished 2024-02-14T17:52:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:04 smithi100 ceph-mon[39557]: osdmap e324: 8 total, 8 up, 8 in 2024-02-14T17:52:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:04 smithi100 ceph-mon[39557]: pgmap v448: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-14T17:52:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:05 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:05 smithi179 ceph-mon[33633]: osdmap e325: 8 total, 8 up, 8 in 2024-02-14T17:52:05.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:05 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:05.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:05 smithi100 ceph-mon[30138]: osdmap e325: 8 total, 8 up, 8 in 2024-02-14T17:52:05.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:05 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:05.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:05 smithi100 ceph-mon[39557]: osdmap e325: 8 total, 8 up, 8 in 2024-02-14T17:52:06.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[30138]: osdmap e326: 8 total, 8 up, 8 in 2024-02-14T17:52:06.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:06.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:06.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:06.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:06.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[30138]: pgmap v451: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:06.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[39557]: osdmap e326: 8 total, 8 up, 8 in 2024-02-14T17:52:06.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:06.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:06.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:06.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:06.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:06 smithi100 ceph-mon[39557]: pgmap v451: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:06 smithi179 ceph-mon[33633]: osdmap e326: 8 total, 8 up, 8 in 2024-02-14T17:52:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:07.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:07.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:06 smithi179 ceph-mon[33633]: pgmap v451: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:07.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]': finished 2024-02-14T17:52:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[30138]: osdmap e327: 8 total, 8 up, 8 in 2024-02-14T17:52:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:07.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:07.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]': finished 2024-02-14T17:52:07.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:07.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[39557]: osdmap e327: 8 total, 8 up, 8 in 2024-02-14T17:52:07.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:07.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:07.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:08.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:08.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi100-79612-61"}]': finished 2024-02-14T17:52:08.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/294205512' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:08.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:07 smithi179 ceph-mon[33633]: osdmap e327: 8 total, 8 up, 8 in 2024-02-14T17:52:08.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]: dispatch 2024-02-14T17:52:08.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:08.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]': finished 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: osdmap e328: 8 total, 8 up, 8 in 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: pgmap v454: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-47"}]': finished 2024-02-14T17:52:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: osdmap e329: 8 total, 8 up, 8 in 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-47", "mode": "writeback"}]: dispatch 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-79612-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-47", "mode": "writeback"}]: dispatch 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]': finished 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: osdmap e328: 8 total, 8 up, 8 in 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:08.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: pgmap v454: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-47"}]': finished 2024-02-14T17:52:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: osdmap e329: 8 total, 8 up, 8 in 2024-02-14T17:52:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-47", "mode": "writeback"}]: dispatch 2024-02-14T17:52:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-79612-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:08.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-47", "mode": "writeback"}]: dispatch 2024-02-14T17:52:09.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi100-79612-61"}]': finished 2024-02-14T17:52:09.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: osdmap e328: 8 total, 8 up, 8 in 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: pgmap v454: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-47"}]': finished 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: osdmap e329: 8 total, 8 up, 8 in 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-47", "mode": "writeback"}]: dispatch 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-79612-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-47", "mode": "writeback"}]: dispatch 2024-02-14T17:52:10.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:09 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:52:10.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-47", "mode": "writeback"}]': finished 2024-02-14T17:52:10.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:09 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:52:10.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-47", "mode": "writeback"}]': finished 2024-02-14T17:52:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:09 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:52:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-47", "mode": "writeback"}]': finished 2024-02-14T17:52:10.829 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:52:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[30138]: pgmap v457: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[30138]: osdmap e330: 8 total, 8 up, 8 in 2024-02-14T17:52:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-79612-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-79612-62"}]': finished 2024-02-14T17:52:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:52:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[30138]: osdmap e331: 8 total, 8 up, 8 in 2024-02-14T17:52:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:11.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[39557]: pgmap v457: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[39557]: osdmap e330: 8 total, 8 up, 8 in 2024-02-14T17:52:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-79612-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-79612-62"}]': finished 2024-02-14T17:52:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:52:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[39557]: osdmap e331: 8 total, 8 up, 8 in 2024-02-14T17:52:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:11.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:10 smithi179 ceph-mon[33633]: pgmap v457: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:10 smithi179 ceph-mon[33633]: osdmap e330: 8 total, 8 up, 8 in 2024-02-14T17:52:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi100-79612-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi100-79612-62"}]': finished 2024-02-14T17:52:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:52:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:10 smithi179 ceph-mon[33633]: osdmap e331: 8 total, 8 up, 8 in 2024-02-14T17:52:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:12.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:52:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:52:12.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[30138]: osdmap e332: 8 total, 8 up, 8 in 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[30138]: pgmap v460: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[39557]: osdmap e332: 8 total, 8 up, 8 in 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[39557]: pgmap v460: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:52:12.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:12 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:12 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:12 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:52:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:12 smithi179 ceph-mon[33633]: osdmap e332: 8 total, 8 up, 8 in 2024-02-14T17:52:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:12 smithi179 ceph-mon[33633]: pgmap v460: 332 pgs: 8 creating+peering, 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:52:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:12 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:52:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:52:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[30138]: osdmap e333: 8 total, 8 up, 8 in 2024-02-14T17:52:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:52:13.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:52:13.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[39557]: osdmap e333: 8 total, 8 up, 8 in 2024-02-14T17:52:13.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:13.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:13.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:13 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:13 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:52:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:52:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:13 smithi179 ceph-mon[33633]: osdmap e333: 8 total, 8 up, 8 in 2024-02-14T17:52:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:14.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62"}]': finished 2024-02-14T17:52:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:52:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[39557]: osdmap e334: 8 total, 8 up, 8 in 2024-02-14T17:52:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:52:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:52:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[39557]: pgmap v463: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:52:14.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62"}]': finished 2024-02-14T17:52:14.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:52:14.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[30138]: osdmap e334: 8 total, 8 up, 8 in 2024-02-14T17:52:14.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:52:14.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:14.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:52:14.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:14 smithi100 ceph-mon[30138]: pgmap v463: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:52:15.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi100-79612-62"}]': finished 2024-02-14T17:52:15.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:52:15.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:14 smithi179 ceph-mon[33633]: osdmap e334: 8 total, 8 up, 8 in 2024-02-14T17:52:15.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:52:15.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-79612-62"}]: dispatch 2024-02-14T17:52:15.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T17:52:15.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:14 smithi179 ceph-mon[33633]: pgmap v463: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T17:52:15.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-79612-62"}]': finished 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[39557]: osdmap e335: 8 total, 8 up, 8 in 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-79612-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-79612-62"}]': finished 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[30138]: osdmap e335: 8 total, 8 up, 8 in 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:52:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:52:15.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:15.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:15.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-79612-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:16.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2858660136' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi100-79612-62"}]': finished 2024-02-14T17:52:16.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T17:52:16.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:15 smithi179 ceph-mon[33633]: osdmap e335: 8 total, 8 up, 8 in 2024-02-14T17:52:16.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:52:16.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T17:52:16.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:16.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:16.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-79612-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:16.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T17:52:16.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-79612-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[30138]: osdmap e336: 8 total, 8 up, 8 in 2024-02-14T17:52:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-79612-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[30138]: pgmap v466: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:16.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T17:52:16.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-79612-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:16.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[39557]: osdmap e336: 8 total, 8 up, 8 in 2024-02-14T17:52:16.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-79612-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:16.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[39557]: pgmap v466: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:16.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:16.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:17.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T17:52:17.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi100-79612-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:17.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:16 smithi179 ceph-mon[33633]: osdmap e336: 8 total, 8 up, 8 in 2024-02-14T17:52:17.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-79612-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:17.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:16 smithi179 ceph-mon[33633]: pgmap v466: 324 pgs: 324 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:17.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:17.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:17.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:52:17.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:17 smithi100 ceph-mon[39557]: osdmap e337: 8 total, 8 up, 8 in 2024-02-14T17:52:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:52:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:17 smithi100 ceph-mon[30138]: osdmap e337: 8 total, 8 up, 8 in 2024-02-14T17:52:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:18.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:52:18.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:18.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:17 smithi179 ceph-mon[33633]: osdmap e337: 8 total, 8 up, 8 in 2024-02-14T17:52:18.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-79612-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-79612-63"}]': finished 2024-02-14T17:52:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]': finished 2024-02-14T17:52:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[30138]: osdmap e338: 8 total, 8 up, 8 in 2024-02-14T17:52:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[30138]: pgmap v469: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:52:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:18.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-79612-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-79612-63"}]': finished 2024-02-14T17:52:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]': finished 2024-02-14T17:52:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[39557]: osdmap e338: 8 total, 8 up, 8 in 2024-02-14T17:52:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[39557]: pgmap v469: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:52:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:18 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:19.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi100-79612-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi100-79612-63"}]': finished 2024-02-14T17:52:19.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]': finished 2024-02-14T17:52:19.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:18 smithi179 ceph-mon[33633]: osdmap e338: 8 total, 8 up, 8 in 2024-02-14T17:52:19.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:18 smithi179 ceph-mon[33633]: pgmap v469: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:52:19.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:19.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:19.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:19.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-47"}]: dispatch 2024-02-14T17:52:19.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:18 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:19.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:19 smithi100 ceph-mon[39557]: osdmap e339: 8 total, 8 up, 8 in 2024-02-14T17:52:19.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:19 smithi100 ceph-mon[30138]: osdmap e339: 8 total, 8 up, 8 in 2024-02-14T17:52:20.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:19 smithi179 ceph-mon[33633]: osdmap e339: 8 total, 8 up, 8 in 2024-02-14T17:52:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:52:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[30138]: pgmap v471: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:52:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[30138]: osdmap e340: 8 total, 8 up, 8 in 2024-02-14T17:52:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:52:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[39557]: pgmap v471: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:52:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[39557]: osdmap e340: 8 total, 8 up, 8 in 2024-02-14T17:52:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:20.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:20.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:20.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:20 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:52:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:20 smithi179 ceph-mon[33633]: pgmap v471: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T17:52:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:20 smithi179 ceph-mon[33633]: osdmap e340: 8 total, 8 up, 8 in 2024-02-14T17:52:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:21.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:20 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:52:22.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:52:22.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-79612-63"}]': finished 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: osdmap e341: 8 total, 8 up, 8 in 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-79612-63"}]': finished 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: osdmap e342: 8 total, 8 up, 8 in 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:22.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-79612-63"}]': finished 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: osdmap e341: 8 total, 8 up, 8 in 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-79612-63"}]': finished 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: osdmap e342: 8 total, 8 up, 8 in 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:22.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi100-79612-63"}]': finished 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: osdmap e341: 8 total, 8 up, 8 in 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-79612-63"}]: dispatch 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3202480842' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi100-79612-63"}]': finished 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: osdmap e342: 8 total, 8 up, 8 in 2024-02-14T17:52:22.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:22.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:23.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: pgmap v474: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:52:23.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:23.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:23.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-79612-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-49"}]': finished 2024-02-14T17:52:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-79612-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: osdmap e343: 8 total, 8 up, 8 in 2024-02-14T17:52:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:52:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-79612-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:23.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:52:23.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: pgmap v474: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:52:23.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:23.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:23.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-79612-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:23.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-49"}]': finished 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-79612-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: osdmap e343: 8 total, 8 up, 8 in 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-79612-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: pgmap v474: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-79612-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:23.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-49"}]': finished 2024-02-14T17:52:23.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi100-79612-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:23.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: osdmap e343: 8 total, 8 up, 8 in 2024-02-14T17:52:23.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:52:23.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-79612-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:23.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-49", "mode": "readproxy"}]: dispatch 2024-02-14T17:52:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:23 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:52:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:23 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-49", "mode": "readproxy"}]': finished 2024-02-14T17:52:24.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:23 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:52:24.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:23 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-49", "mode": "readproxy"}]': finished 2024-02-14T17:52:24.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:23 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:52:24.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:23 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-49", "mode": "readproxy"}]': finished 2024-02-14T17:52:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:24 smithi179 ceph-mon[33633]: pgmap v477: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:52:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:24 smithi179 ceph-mon[33633]: osdmap e344: 8 total, 8 up, 8 in 2024-02-14T17:52:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-79612-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-79612-64"}]': finished 2024-02-14T17:52:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:24 smithi179 ceph-mon[33633]: osdmap e345: 8 total, 8 up, 8 in 2024-02-14T17:52:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:24 smithi100 ceph-mon[30138]: pgmap v477: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:52:25.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:24 smithi100 ceph-mon[30138]: osdmap e344: 8 total, 8 up, 8 in 2024-02-14T17:52:25.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-79612-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-79612-64"}]': finished 2024-02-14T17:52:25.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:24 smithi100 ceph-mon[30138]: osdmap e345: 8 total, 8 up, 8 in 2024-02-14T17:52:25.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:24 smithi100 ceph-mon[39557]: pgmap v477: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:52:25.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:24 smithi100 ceph-mon[39557]: osdmap e344: 8 total, 8 up, 8 in 2024-02-14T17:52:25.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi100-79612-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi100-79612-64"}]': finished 2024-02-14T17:52:25.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:24 smithi100 ceph-mon[39557]: osdmap e345: 8 total, 8 up, 8 in 2024-02-14T17:52:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:26 smithi179 ceph-mon[33633]: pgmap v480: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:26 smithi179 ceph-mon[33633]: osdmap e346: 8 total, 8 up, 8 in 2024-02-14T17:52:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:26 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:26 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:27.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:26 smithi100 ceph-mon[39557]: pgmap v480: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:27.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:26 smithi100 ceph-mon[39557]: osdmap e346: 8 total, 8 up, 8 in 2024-02-14T17:52:27.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:26 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:27.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:26 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:26 smithi100 ceph-mon[30138]: pgmap v480: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:26 smithi100 ceph-mon[30138]: osdmap e346: 8 total, 8 up, 8 in 2024-02-14T17:52:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:26 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:26 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:27 smithi179 ceph-mon[33633]: osdmap e347: 8 total, 8 up, 8 in 2024-02-14T17:52:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-79612-64"}]': finished 2024-02-14T17:52:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:27 smithi179 ceph-mon[33633]: osdmap e348: 8 total, 8 up, 8 in 2024-02-14T17:52:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:28.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:27 smithi100 ceph-mon[39557]: osdmap e347: 8 total, 8 up, 8 in 2024-02-14T17:52:28.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:28.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-79612-64"}]': finished 2024-02-14T17:52:28.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:27 smithi100 ceph-mon[39557]: osdmap e348: 8 total, 8 up, 8 in 2024-02-14T17:52:28.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:27 smithi100 ceph-mon[30138]: osdmap e347: 8 total, 8 up, 8 in 2024-02-14T17:52:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi100-79612-64"}]': finished 2024-02-14T17:52:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:27 smithi100 ceph-mon[30138]: osdmap e348: 8 total, 8 up, 8 in 2024-02-14T17:52:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-79612-64"}]: dispatch 2024-02-14T17:52:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:28 smithi179 ceph-mon[33633]: pgmap v483: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:52:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-79612-64"}]': finished 2024-02-14T17:52:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:28 smithi179 ceph-mon[33633]: osdmap e349: 8 total, 8 up, 8 in 2024-02-14T17:52:29.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:28 smithi100 ceph-mon[39557]: pgmap v483: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:52:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-79612-64"}]': finished 2024-02-14T17:52:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:28 smithi100 ceph-mon[39557]: osdmap e349: 8 total, 8 up, 8 in 2024-02-14T17:52:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:28 smithi100 ceph-mon[30138]: pgmap v483: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:52:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1586432309' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi100-79612-64"}]': finished 2024-02-14T17:52:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:28 smithi100 ceph-mon[30138]: osdmap e349: 8 total, 8 up, 8 in 2024-02-14T17:52:30.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-79612-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:30.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-79612-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-79612-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-79612-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-79612-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:30.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-79612-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:52:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:30 smithi179 ceph-mon[33633]: pgmap v486: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:52:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-79612-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:30 smithi179 ceph-mon[33633]: osdmap e350: 8 total, 8 up, 8 in 2024-02-14T17:52:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-79612-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:31.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-79612-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:31.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-mon[39557]: pgmap v486: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:52:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-79612-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-mon[39557]: osdmap e350: 8 total, 8 up, 8 in 2024-02-14T17:52:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-79612-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-79612-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-mon[30138]: pgmap v486: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-14T17:52:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi100-79612-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-mon[30138]: osdmap e350: 8 total, 8 up, 8 in 2024-02-14T17:52:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-79612-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-79612-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:32.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:52:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:52:32.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:31 smithi179 ceph-mon[33633]: osdmap e351: 8 total, 8 up, 8 in 2024-02-14T17:52:32.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:31 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-79612-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-79612-65"}]': finished 2024-02-14T17:52:32.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:31 smithi179 ceph-mon[33633]: osdmap e352: 8 total, 8 up, 8 in 2024-02-14T17:52:32.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:31 smithi100 ceph-mon[39557]: osdmap e351: 8 total, 8 up, 8 in 2024-02-14T17:52:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:31 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-79612-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-79612-65"}]': finished 2024-02-14T17:52:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:31 smithi100 ceph-mon[39557]: osdmap e352: 8 total, 8 up, 8 in 2024-02-14T17:52:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:31 smithi100 ceph-mon[30138]: osdmap e351: 8 total, 8 up, 8 in 2024-02-14T17:52:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:31 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi100-79612-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi100-79612-65"}]': finished 2024-02-14T17:52:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:31 smithi100 ceph-mon[30138]: osdmap e352: 8 total, 8 up, 8 in 2024-02-14T17:52:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:32 smithi179 ceph-mon[33633]: pgmap v489: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:32 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:32 smithi179 ceph-mon[33633]: osdmap e353: 8 total, 8 up, 8 in 2024-02-14T17:52:33.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:32 smithi100 ceph-mon[39557]: pgmap v489: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:33.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:32 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:33.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:32 smithi100 ceph-mon[39557]: osdmap e353: 8 total, 8 up, 8 in 2024-02-14T17:52:33.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:32 smithi100 ceph-mon[30138]: pgmap v489: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:33.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:32 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:33.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:32 smithi100 ceph-mon[30138]: osdmap e353: 8 total, 8 up, 8 in 2024-02-14T17:52:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:33 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:33 smithi179 ceph-mon[33633]: osdmap e354: 8 total, 8 up, 8 in 2024-02-14T17:52:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:34.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:33 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:33 smithi100 ceph-mon[39557]: osdmap e354: 8 total, 8 up, 8 in 2024-02-14T17:52:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:33 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:33 smithi100 ceph-mon[30138]: osdmap e354: 8 total, 8 up, 8 in 2024-02-14T17:52:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:34 smithi179 ceph-mon[33633]: pgmap v492: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]': finished 2024-02-14T17:52:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:52:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:34 smithi179 ceph-mon[33633]: osdmap e355: 8 total, 8 up, 8 in 2024-02-14T17:52:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[30138]: pgmap v492: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]': finished 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[30138]: osdmap e355: 8 total, 8 up, 8 in 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[39557]: pgmap v492: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi100-79612-65"}]': finished 2024-02-14T17:52:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:52:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[39557]: osdmap e355: 8 total, 8 up, 8 in 2024-02-14T17:52:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/22590023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]: dispatch 2024-02-14T17:52:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:52:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]': finished 2024-02-14T17:52:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]': finished 2024-02-14T17:52:36.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: osdmap e356: 8 total, 8 up, 8 in 2024-02-14T17:52:36.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:36.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:36.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-79612-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:36.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:36.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:36.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:36.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:35 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:36.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:52:36.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]': finished 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]': finished 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: osdmap e356: 8 total, 8 up, 8 in 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-79612-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]': finished 2024-02-14T17:52:36.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi100-79612-65"}]': finished 2024-02-14T17:52:36.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: osdmap e356: 8 total, 8 up, 8 in 2024-02-14T17:52:36.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:36.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:36.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-79612-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:52:36.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:36.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:36.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:36.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:35 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-49"}]: dispatch 2024-02-14T17:52:36.713 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2163:head 2024-02-14T17:52:36.713 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:5d165639:::164:head 2024-02-14T17:52:36.713 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:f43765fc:::165:head 2024-02-14T17:52:36.713 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:b4c720e9:::166:head 2024-02-14T17:52:36.713 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:e694b040:::167:head 2024-02-14T17:52:36.713 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:afa38db2:::168:head 2024-02-14T17:52:36.713 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:77ba9f53:::169:head 2024-02-14T17:52:36.713 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:87495034:::170:head 2024-02-14T17:52:36.713 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:7c96bf0e:::171:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:dbe346cc:::172:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:e943ec24:::173:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:f97a9c0c:::174:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:6f26e74d:::175:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:4f95e106:::176:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:0e6f2f8f:::177:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:05db05f1:::178:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:38a78d66:::179:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:d095610b:::180:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:a1a9d709:::181:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:1e5d39db:::182:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:f7df4fb9:::183:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:03a7f161:::184:head 2024-02-14T17:52:36.714 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:ba70721e:::185:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:28e5662d:::186:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:973d52de:::187:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:4303eb1c:::188:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:b990b48e:::189:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:29b8165b:::190:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:3547f197:::191:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:7e260936:::192:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:1abec7b1:::193:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:10fdda93:::194:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:15817eea:::195:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:770bab57:::196:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:ed9e13e7:::197:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:71471a8f:::198:head 2024-02-14T17:52:36.715 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: checking for 275:10fb1d02:::199:head 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8065 ms) 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,0 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: first is 1707933112 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,0 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,0 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,1707933114,1707933115,0 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,1707933114,1707933115,0 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,1707933114,1707933115,0 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,1707933114,1707933115,1707933117,1707933118,0 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,1707933114,1707933115,1707933117,1707933118,0 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,1707933114,1707933115,1707933117,1707933118,0 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,1707933114,1707933115,1707933117,1707933118,1707933120,1707933121,0 2024-02-14T17:52:36.716 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,1707933114,1707933115,1707933117,1707933118,1707933120,1707933121,0 2024-02-14T17:52:36.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933112,1707933114,1707933115,1707933117,1707933118,1707933120,1707933121,0 2024-02-14T17:52:36.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933115,1707933117,1707933118,1707933120,1707933121,1707933123,1707933124,0 2024-02-14T17:52:36.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: first now 1707933115, trimmed 2024-02-14T17:52:36.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20114 ms) 2024-02-14T17:52:36.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-14T17:52:36.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: foo0 2024-02-14T17:52:36.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-14T17:52:36.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14069 ms) 2024-02-14T17:52:36.717 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-14T17:52:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:36 smithi179 ceph-mon[33633]: pgmap v495: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-79612-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:36 smithi179 ceph-mon[33633]: osdmap e357: 8 total, 8 up, 8 in 2024-02-14T17:52:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-79612-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:36 smithi100 ceph-mon[30138]: pgmap v495: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-79612-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:36 smithi100 ceph-mon[30138]: osdmap e357: 8 total, 8 up, 8 in 2024-02-14T17:52:37.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-79612-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:36 smithi100 ceph-mon[39557]: pgmap v495: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi100-79612-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:52:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:36 smithi100 ceph-mon[39557]: osdmap e357: 8 total, 8 up, 8 in 2024-02-14T17:52:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-79612-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:38 smithi179 ceph-mon[33633]: pgmap v498: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:38 smithi179 ceph-mon[33633]: osdmap e358: 8 total, 8 up, 8 in 2024-02-14T17:52:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:39.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:39.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:38 smithi100 ceph-mon[39557]: pgmap v498: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:39.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:38 smithi100 ceph-mon[39557]: osdmap e358: 8 total, 8 up, 8 in 2024-02-14T17:52:39.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:38 smithi100 ceph-mon[30138]: pgmap v498: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:38 smithi100 ceph-mon[30138]: osdmap e358: 8 total, 8 up, 8 in 2024-02-14T17:52:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:52:40.051 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-79612-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-79612-66"}]': finished 2024-02-14T17:52:40.052 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:40.052 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:39 smithi100 ceph-mon[30138]: osdmap e359: 8 total, 8 up, 8 in 2024-02-14T17:52:40.052 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:40.052 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:40.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-79612-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-79612-66"}]': finished 2024-02-14T17:52:40.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:40.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:39 smithi100 ceph-mon[39557]: osdmap e359: 8 total, 8 up, 8 in 2024-02-14T17:52:40.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:40.052 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi100-79612-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi100-79612-66"}]': finished 2024-02-14T17:52:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:52:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:39 smithi179 ceph-mon[33633]: osdmap e359: 8 total, 8 up, 8 in 2024-02-14T17:52:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:40.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:52:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:52:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:40.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[30138]: pgmap v501: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:40.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:40.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:52:40.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[30138]: osdmap e360: 8 total, 8 up, 8 in 2024-02-14T17:52:40.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:40.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:40.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[39557]: pgmap v501: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:40.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:40.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:52:40.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[39557]: osdmap e360: 8 total, 8 up, 8 in 2024-02-14T17:52:40.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:40.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:41.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:40 smithi179 ceph-mon[33633]: pgmap v501: 332 pgs: 40 unknown, 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:52:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:40 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:52:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:40 smithi179 ceph-mon[33633]: osdmap e360: 8 total, 8 up, 8 in 2024-02-14T17:52:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:41.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:42.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-51"}]': finished 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: osdmap e361: 8 total, 8 up, 8 in 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-51", "mode": "writeback"}]: dispatch 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-51", "mode": "writeback"}]: dispatch 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-51", "mode": "writeback"}]': finished 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-79612-66"}]': finished 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: osdmap e362: 8 total, 8 up, 8 in 2024-02-14T17:52:42.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:41 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:42.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-51"}]': finished 2024-02-14T17:52:42.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: osdmap e361: 8 total, 8 up, 8 in 2024-02-14T17:52:42.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-51", "mode": "writeback"}]: dispatch 2024-02-14T17:52:42.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-51", "mode": "writeback"}]: dispatch 2024-02-14T17:52:42.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:42.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:42.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:42.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:52:42.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-51", "mode": "writeback"}]': finished 2024-02-14T17:52:42.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-79612-66"}]': finished 2024-02-14T17:52:42.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: osdmap e362: 8 total, 8 up, 8 in 2024-02-14T17:52:42.152 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-51"}]': finished 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: osdmap e361: 8 total, 8 up, 8 in 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-51", "mode": "writeback"}]: dispatch 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-51", "mode": "writeback"}]: dispatch 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-51", "mode": "writeback"}]': finished 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi100-79612-66"}]': finished 2024-02-14T17:52:42.153 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: osdmap e362: 8 total, 8 up, 8 in 2024-02-14T17:52:42.154 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:41 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-79612-66"}]: dispatch 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3037 ms) 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7045 ms) 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7026 ms) 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7046 ms) 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7015 ms) 2024-02-14T17:52:42.736 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-14T17:52:42.737 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7031 ms) 2024-02-14T17:52:42.737 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-14T17:52:42.737 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7023 ms) 2024-02-14T17:52:42.737 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139418 ms total) 2024-02-14T17:52:42.737 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: 2024-02-14T17:52:42.737 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-14T17:52:42.737 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (296738 ms total) 2024-02-14T17:52:42.737 INFO:tasks.workunit.client.0.smithi100.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-02-14T17:52:43.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:42 smithi179 ceph-mon[33633]: pgmap v504: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:52:43.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:43.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:43.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:42 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:43.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-79612-66"}]': finished 2024-02-14T17:52:43.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:52:43.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:42 smithi179 ceph-mon[33633]: osdmap e363: 8 total, 8 up, 8 in 2024-02-14T17:52:43.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:43.073 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:43.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[30138]: pgmap v504: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-79612-66"}]': finished 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[30138]: osdmap e363: 8 total, 8 up, 8 in 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[39557]: pgmap v504: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1795841066' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi100-79612-66"}]': finished 2024-02-14T17:52:43.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:52:43.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[39557]: osdmap e363: 8 total, 8 up, 8 in 2024-02-14T17:52:43.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:43.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:52:45.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:44 smithi179 ceph-mon[33633]: pgmap v507: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:52:45.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:52:45.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:44 smithi179 ceph-mon[33633]: osdmap e364: 8 total, 8 up, 8 in 2024-02-14T17:52:45.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:45.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:45.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:44 smithi100 ceph-mon[30138]: pgmap v507: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:52:45.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:52:45.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:44 smithi100 ceph-mon[30138]: osdmap e364: 8 total, 8 up, 8 in 2024-02-14T17:52:45.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:45.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:45.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:44 smithi100 ceph-mon[39557]: pgmap v507: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T17:52:45.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:52:45.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:44 smithi100 ceph-mon[39557]: osdmap e364: 8 total, 8 up, 8 in 2024-02-14T17:52:45.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:45.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:52:46.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:45 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:52:46.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:52:46.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:45 smithi179 ceph-mon[33633]: osdmap e365: 8 total, 8 up, 8 in 2024-02-14T17:52:46.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:46.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:46.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:52:46.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:45 smithi179 ceph-mon[33633]: osdmap e366: 8 total, 8 up, 8 in 2024-02-14T17:52:46.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:52:46.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:52:46.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[30138]: osdmap e365: 8 total, 8 up, 8 in 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[30138]: osdmap e366: 8 total, 8 up, 8 in 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:52:46.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[39557]: osdmap e365: 8 total, 8 up, 8 in 2024-02-14T17:52:46.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:46.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T17:52:46.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T17:52:46.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[39557]: osdmap e366: 8 total, 8 up, 8 in 2024-02-14T17:52:46.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:52:46.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T17:52:47.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:46 smithi179 ceph-mon[33633]: pgmap v510: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:47.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "target_max_objects","val": "1"}]': finished 2024-02-14T17:52:47.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:46 smithi179 ceph-mon[33633]: osdmap e367: 8 total, 8 up, 8 in 2024-02-14T17:52:47.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:46 smithi100 ceph-mon[30138]: pgmap v510: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:47.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "target_max_objects","val": "1"}]': finished 2024-02-14T17:52:47.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:46 smithi100 ceph-mon[30138]: osdmap e367: 8 total, 8 up, 8 in 2024-02-14T17:52:47.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:46 smithi100 ceph-mon[39557]: pgmap v510: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:52:47.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-51","var": "target_max_objects","val": "1"}]': finished 2024-02-14T17:52:47.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:46 smithi100 ceph-mon[39557]: osdmap e367: 8 total, 8 up, 8 in 2024-02-14T17:52:48.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:47 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T17:52:48.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:47 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T17:52:48.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:47 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T17:52:49.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:48 smithi179 ceph-mon[33633]: pgmap v513: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T17:52:49.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:48 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:49.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:48 smithi100 ceph-mon[30138]: pgmap v513: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T17:52:49.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:48 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:49.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:48 smithi100 ceph-mon[39557]: pgmap v513: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T17:52:49.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:48 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:52:50.805 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:52:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:52:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:52:51.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:50 smithi179 ceph-mon[33633]: pgmap v514: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:51.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:50 smithi100 ceph-mon[30138]: pgmap v514: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:51.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:50 smithi100 ceph-mon[39557]: pgmap v514: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:52.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:52:51 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:52:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:52 smithi179 ceph-mon[33633]: pgmap v515: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T17:52:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:52 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:53.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:52 smithi100 ceph-mon[30138]: pgmap v515: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T17:52:53.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:52 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:53.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:52 smithi100 ceph-mon[39557]: pgmap v515: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T17:52:53.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:52 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:52:54.481 INFO:tasks.workunit.client.0.smithi100.stdout:: started 0x5648c4778a10 2024-02-14T17:52:54.481 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4778b20 2024-02-14T17:52:54.481 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4778ed0 2024-02-14T17:52:54.481 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4779000 2024-02-14T17:52:54.481 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47795f0 2024-02-14T17:52:54.481 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4779150 2024-02-14T17:52:54.481 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47792a0 2024-02-14T17:52:54.481 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47793d0 2024-02-14T17:52:54.481 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47794e0 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4779ba0 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4779700 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4779830 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4779940 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4779a50 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477a170 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4779cd0 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4779e00 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4779f10 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477a040 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477a720 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477a280 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477a3b0 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477a4e0 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477a850 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477acc0 2024-02-14T17:52:54.482 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477adf0 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477b3c0 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477a960 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477a610 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477aa90 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477b510 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477af20 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477b050 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477aba0 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477b160 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477b270 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477b640 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477bab0 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477bc00 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4778580 2024-02-14T17:52:54.483 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47786d0 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477c1f0 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477bd50 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477be80 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477bfb0 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477c0e0 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477c7e0 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477c340 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477c490 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477c5a0 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477c930 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477cde0 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477cef0 2024-02-14T17:52:54.484 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477c6b0 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477ca80 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477cbb0 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477d000 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477d490 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477d5c0 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477b750 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477b880 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477db90 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477d6f0 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477d820 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477d950 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477da80 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477e140 2024-02-14T17:52:54.485 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477dca0 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477ddb0 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477dee0 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477e030 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477e6f0 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477e250 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477e380 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477e490 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477e5e0 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477ecc0 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477e820 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477e950 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477ea60 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477ebb0 2024-02-14T17:52:54.486 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477f290 2024-02-14T17:52:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:54 smithi179 ceph-mon[33633]: pgmap v516: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:55.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:54 smithi100 ceph-mon[39557]: pgmap v516: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:55.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:54 smithi100 ceph-mon[30138]: pgmap v516: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:56 smithi179 ceph-mon[33633]: pgmap v517: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:56 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:56 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:57.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:56 smithi100 ceph-mon[39557]: pgmap v517: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:57.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:56 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:57.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:56 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:57.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:57.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:56 smithi100 ceph-mon[30138]: pgmap v517: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:52:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:56 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:52:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:56 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:52:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:52:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:52:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:58 smithi179 ceph-mon[33633]: osdmap e368: 8 total, 8 up, 8 in 2024-02-14T17:52:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:58.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:58 smithi179 ceph-mon[33633]: pgmap v519: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:52:58.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:52:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:58 smithi100 ceph-mon[39557]: osdmap e368: 8 total, 8 up, 8 in 2024-02-14T17:52:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:58 smithi100 ceph-mon[39557]: pgmap v519: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:52:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:52:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:58 smithi100 ceph-mon[30138]: osdmap e368: 8 total, 8 up, 8 in 2024-02-14T17:52:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:52:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:58 smithi100 ceph-mon[30138]: pgmap v519: 324 pgs: 324 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:00.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]': finished 2024-02-14T17:53:00.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:59 smithi179 ceph-mon[33633]: osdmap e369: 8 total, 8 up, 8 in 2024-02-14T17:53:00.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:00.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:00.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:53:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:52:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:53:00.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]': finished 2024-02-14T17:53:00.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[39557]: osdmap e369: 8 total, 8 up, 8 in 2024-02-14T17:53:00.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:53:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:53:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]': finished 2024-02-14T17:53:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[30138]: osdmap e369: 8 total, 8 up, 8 in 2024-02-14T17:53:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:53:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:52:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-51"}]: dispatch 2024-02-14T17:53:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:53:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-mon[30138]: osdmap e370: 8 total, 8 up, 8 in 2024-02-14T17:53:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-mon[30138]: pgmap v522: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-14T17:53:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-mon[30138]: osdmap e371: 8 total, 8 up, 8 in 2024-02-14T17:53:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:00.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-mon[39557]: osdmap e370: 8 total, 8 up, 8 in 2024-02-14T17:53:00.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-mon[39557]: pgmap v522: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-14T17:53:00.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-mon[39557]: osdmap e371: 8 total, 8 up, 8 in 2024-02-14T17:53:00.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:00.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:00 smithi179 ceph-mon[33633]: osdmap e370: 8 total, 8 up, 8 in 2024-02-14T17:53:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:00 smithi179 ceph-mon[33633]: pgmap v522: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-14T17:53:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:00 smithi179 ceph-mon[33633]: osdmap e371: 8 total, 8 up, 8 in 2024-02-14T17:53:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:01.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:01.955 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:01 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T17:53:01.956 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:01.956 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:01 smithi179 ceph-mon[33633]: osdmap e372: 8 total, 8 up, 8 in 2024-02-14T17:53:01.956 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:01.956 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:01.956 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-53"}]: dispatch 2024-02-14T17:53:01.956 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-53"}]: dispatch 2024-02-14T17:53:02.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[30138]: osdmap e372: 8 total, 8 up, 8 in 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-53"}]: dispatch 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-53"}]: dispatch 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[39557]: osdmap e372: 8 total, 8 up, 8 in 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-53"}]: dispatch 2024-02-14T17:53:02.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-53"}]: dispatch 2024-02-14T17:53:02.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:53:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:53:03.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:02 smithi179 ceph-mon[33633]: pgmap v525: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:53:03.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:02 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:03.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:02 smithi179 ceph-mon[33633]: osdmap e373: 8 total, 8 up, 8 in 2024-02-14T17:53:03.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:02 smithi100 ceph-mon[30138]: pgmap v525: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:53:03.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:02 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:03.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:02 smithi100 ceph-mon[30138]: osdmap e373: 8 total, 8 up, 8 in 2024-02-14T17:53:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:02 smithi100 ceph-mon[39557]: pgmap v525: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T17:53:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:02 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:02 smithi100 ceph-mon[39557]: osdmap e373: 8 total, 8 up, 8 in 2024-02-14T17:53:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:04 smithi179 ceph-mon[33633]: osdmap e374: 8 total, 8 up, 8 in 2024-02-14T17:53:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:04.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:04 smithi179 ceph-mon[33633]: pgmap v528: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:53:04.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:04 smithi100 ceph-mon[30138]: osdmap e374: 8 total, 8 up, 8 in 2024-02-14T17:53:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:04 smithi100 ceph-mon[30138]: pgmap v528: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:53:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:04 smithi100 ceph-mon[39557]: osdmap e374: 8 total, 8 up, 8 in 2024-02-14T17:53:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:04 smithi100 ceph-mon[39557]: pgmap v528: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:53:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:05 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:05 smithi179 ceph-mon[33633]: osdmap e375: 8 total, 8 up, 8 in 2024-02-14T17:53:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:05.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:05.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-55"}]: dispatch 2024-02-14T17:53:05.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-55"}]: dispatch 2024-02-14T17:53:05.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:05.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[30138]: osdmap e375: 8 total, 8 up, 8 in 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-55"}]: dispatch 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-55"}]: dispatch 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[39557]: osdmap e375: 8 total, 8 up, 8 in 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:05.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:05.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:05.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-55"}]: dispatch 2024-02-14T17:53:05.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-55"}]: dispatch 2024-02-14T17:53:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:06 smithi179 ceph-mon[33633]: osdmap e376: 8 total, 8 up, 8 in 2024-02-14T17:53:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:06 smithi179 ceph-mon[33633]: pgmap v531: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:06.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:06 smithi100 ceph-mon[39557]: osdmap e376: 8 total, 8 up, 8 in 2024-02-14T17:53:06.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:06 smithi100 ceph-mon[39557]: pgmap v531: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:06.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:06 smithi100 ceph-mon[30138]: osdmap e376: 8 total, 8 up, 8 in 2024-02-14T17:53:06.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:06 smithi100 ceph-mon[30138]: pgmap v531: 292 pgs: 292 active+clean; 8.4 MiB data, 645 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:07 smithi179 ceph-mon[33633]: osdmap e377: 8 total, 8 up, 8 in 2024-02-14T17:53:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:07 smithi100 ceph-mon[30138]: osdmap e377: 8 total, 8 up, 8 in 2024-02-14T17:53:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:07.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:07.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:07 smithi100 ceph-mon[39557]: osdmap e377: 8 total, 8 up, 8 in 2024-02-14T17:53:07.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:07.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:08 smithi179 ceph-mon[33633]: osdmap e378: 8 total, 8 up, 8 in 2024-02-14T17:53:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-57"}]: dispatch 2024-02-14T17:53:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-57"}]: dispatch 2024-02-14T17:53:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:08 smithi179 ceph-mon[33633]: pgmap v534: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:08.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:08.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[39557]: osdmap e378: 8 total, 8 up, 8 in 2024-02-14T17:53:08.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:08.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:08.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-57"}]: dispatch 2024-02-14T17:53:08.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-57"}]: dispatch 2024-02-14T17:53:08.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[39557]: pgmap v534: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[30138]: osdmap e378: 8 total, 8 up, 8 in 2024-02-14T17:53:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:08.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-57"}]: dispatch 2024-02-14T17:53:08.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-57"}]: dispatch 2024-02-14T17:53:08.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:08 smithi100 ceph-mon[30138]: pgmap v534: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:09.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:09 smithi179 ceph-mon[33633]: osdmap e379: 8 total, 8 up, 8 in 2024-02-14T17:53:09.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:09 smithi100 ceph-mon[39557]: osdmap e379: 8 total, 8 up, 8 in 2024-02-14T17:53:09.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:09 smithi100 ceph-mon[30138]: osdmap e379: 8 total, 8 up, 8 in 2024-02-14T17:53:10.803 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:53:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:53:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:10.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:10 smithi100 ceph-mon[30138]: osdmap e380: 8 total, 8 up, 8 in 2024-02-14T17:53:10.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:10.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:10.804 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:10 smithi100 ceph-mon[30138]: pgmap v537: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:10.804 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:10 smithi100 ceph-mon[39557]: osdmap e380: 8 total, 8 up, 8 in 2024-02-14T17:53:10.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:10.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:10.805 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:10 smithi100 ceph-mon[39557]: pgmap v537: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:10 smithi179 ceph-mon[33633]: osdmap e380: 8 total, 8 up, 8 in 2024-02-14T17:53:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:10 smithi179 ceph-mon[33633]: pgmap v537: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-mon[33633]: osdmap e381: 8 total, 8 up, 8 in 2024-02-14T17:53:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-59"}]: dispatch 2024-02-14T17:53:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-59"}]: dispatch 2024-02-14T17:53:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:53:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:11.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[30138]: osdmap e381: 8 total, 8 up, 8 in 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-59"}]: dispatch 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-59"}]: dispatch 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:11.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:11.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[39557]: osdmap e381: 8 total, 8 up, 8 in 2024-02-14T17:53:11.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:11.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:11.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:11.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-59"}]: dispatch 2024-02-14T17:53:11.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-59"}]: dispatch 2024-02-14T17:53:11.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:53:11.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:12.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:53:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:53:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:12 smithi179 ceph-mon[33633]: osdmap e382: 8 total, 8 up, 8 in 2024-02-14T17:53:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:12 smithi179 ceph-mon[33633]: pgmap v540: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:12 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:12.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:12 smithi100 ceph-mon[30138]: osdmap e382: 8 total, 8 up, 8 in 2024-02-14T17:53:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:12 smithi100 ceph-mon[30138]: pgmap v540: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:12 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:12 smithi100 ceph-mon[39557]: osdmap e382: 8 total, 8 up, 8 in 2024-02-14T17:53:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:12 smithi100 ceph-mon[39557]: pgmap v540: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:12 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:13 smithi179 ceph-mon[33633]: osdmap e383: 8 total, 8 up, 8 in 2024-02-14T17:53:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:13 smithi100 ceph-mon[30138]: osdmap e383: 8 total, 8 up, 8 in 2024-02-14T17:53:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:13.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:13 smithi100 ceph-mon[39557]: osdmap e383: 8 total, 8 up, 8 in 2024-02-14T17:53:13.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:13.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:14 smithi179 ceph-mon[33633]: osdmap e384: 8 total, 8 up, 8 in 2024-02-14T17:53:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:14 smithi179 ceph-mon[33633]: pgmap v543: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:14.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:14.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[30138]: osdmap e384: 8 total, 8 up, 8 in 2024-02-14T17:53:14.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:14.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:14.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:14.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[30138]: pgmap v543: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:14.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[39557]: osdmap e384: 8 total, 8 up, 8 in 2024-02-14T17:53:14.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:14.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:14.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:14.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:14 smithi100 ceph-mon[39557]: pgmap v543: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:15 smithi179 ceph-mon[33633]: osdmap e385: 8 total, 8 up, 8 in 2024-02-14T17:53:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-61"}]: dispatch 2024-02-14T17:53:15.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-61"}]: dispatch 2024-02-14T17:53:15.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[30138]: osdmap e385: 8 total, 8 up, 8 in 2024-02-14T17:53:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-61"}]: dispatch 2024-02-14T17:53:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-61"}]: dispatch 2024-02-14T17:53:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[39557]: osdmap e385: 8 total, 8 up, 8 in 2024-02-14T17:53:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:15.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-61"}]: dispatch 2024-02-14T17:53:15.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-61"}]: dispatch 2024-02-14T17:53:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:16 smithi179 ceph-mon[33633]: osdmap e386: 8 total, 8 up, 8 in 2024-02-14T17:53:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:16 smithi179 ceph-mon[33633]: pgmap v546: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:16.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:16 smithi100 ceph-mon[30138]: osdmap e386: 8 total, 8 up, 8 in 2024-02-14T17:53:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:16 smithi100 ceph-mon[30138]: pgmap v546: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:16.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:16 smithi100 ceph-mon[39557]: osdmap e386: 8 total, 8 up, 8 in 2024-02-14T17:53:16.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:16 smithi100 ceph-mon[39557]: pgmap v546: 292 pgs: 292 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:17 smithi179 ceph-mon[33633]: osdmap e387: 8 total, 8 up, 8 in 2024-02-14T17:53:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:17.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:17.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:17 smithi100 ceph-mon[30138]: osdmap e387: 8 total, 8 up, 8 in 2024-02-14T17:53:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:17 smithi100 ceph-mon[39557]: osdmap e387: 8 total, 8 up, 8 in 2024-02-14T17:53:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:18 smithi179 ceph-mon[33633]: osdmap e388: 8 total, 8 up, 8 in 2024-02-14T17:53:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:18 smithi179 ceph-mon[33633]: pgmap v549: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:53:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:18 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:18.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[30138]: osdmap e388: 8 total, 8 up, 8 in 2024-02-14T17:53:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[30138]: pgmap v549: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:53:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[39557]: osdmap e388: 8 total, 8 up, 8 in 2024-02-14T17:53:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[39557]: pgmap v549: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:53:18.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:18 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:19.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:19.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:19 smithi179 ceph-mon[33633]: osdmap e389: 8 total, 8 up, 8 in 2024-02-14T17:53:19.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:19.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:19 smithi100 ceph-mon[30138]: osdmap e389: 8 total, 8 up, 8 in 2024-02-14T17:53:19.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:19.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:19 smithi100 ceph-mon[39557]: osdmap e389: 8 total, 8 up, 8 in 2024-02-14T17:53:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:20 smithi179 ceph-mon[33633]: osdmap e390: 8 total, 8 up, 8 in 2024-02-14T17:53:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:20 smithi179 ceph-mon[33633]: pgmap v552: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:53:20.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:20 smithi100 ceph-mon[30138]: osdmap e390: 8 total, 8 up, 8 in 2024-02-14T17:53:20.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:20 smithi100 ceph-mon[30138]: pgmap v552: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:53:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:20 smithi100 ceph-mon[39557]: osdmap e390: 8 total, 8 up, 8 in 2024-02-14T17:53:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:20 smithi100 ceph-mon[39557]: pgmap v552: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:53:20.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:53:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:53:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:21 smithi179 ceph-mon[33633]: osdmap e391: 8 total, 8 up, 8 in 2024-02-14T17:53:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:21 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:53:21.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:21 smithi100 ceph-mon[39557]: osdmap e391: 8 total, 8 up, 8 in 2024-02-14T17:53:21.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:21 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:53:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:21 smithi100 ceph-mon[30138]: osdmap e391: 8 total, 8 up, 8 in 2024-02-14T17:53:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:21 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:53:22.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:53:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:53:22.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:22 smithi179 ceph-mon[33633]: osdmap e392: 8 total, 8 up, 8 in 2024-02-14T17:53:22.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:22 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:53:22.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:22 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:53:22.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:22 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:53:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:22 smithi179 ceph-mon[33633]: pgmap v555: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-14T17:53:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:22 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:22.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[30138]: osdmap e392: 8 total, 8 up, 8 in 2024-02-14T17:53:22.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:53:22.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:53:22.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:53:22.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[30138]: pgmap v555: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-14T17:53:22.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[39557]: osdmap e392: 8 total, 8 up, 8 in 2024-02-14T17:53:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:53:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:53:22.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:53:22.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[39557]: pgmap v555: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-14T17:53:22.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:22 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:23 smithi179 ceph-mon[33633]: osdmap e393: 8 total, 8 up, 8 in 2024-02-14T17:53:23.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:23 smithi100 ceph-mon[30138]: osdmap e393: 8 total, 8 up, 8 in 2024-02-14T17:53:23.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:23 smithi100 ceph-mon[39557]: osdmap e393: 8 total, 8 up, 8 in 2024-02-14T17:53:24.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:24 smithi100 ceph-mon[30138]: pgmap v557: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-14T17:53:24.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:24 smithi100 ceph-mon[39557]: pgmap v557: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-14T17:53:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:24 smithi179 ceph-mon[33633]: pgmap v557: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-14T17:53:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:26 smithi179 ceph-mon[33633]: pgmap v558: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:26 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:27.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:26 smithi100 ceph-mon[39557]: pgmap v558: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:27.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:26 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:26 smithi100 ceph-mon[30138]: pgmap v558: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T17:53:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:26 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:28 smithi179 ceph-mon[33633]: pgmap v559: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:53:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:28 smithi179 ceph-mon[33633]: osdmap e394: 8 total, 8 up, 8 in 2024-02-14T17:53:29.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:28 smithi100 ceph-mon[39557]: pgmap v559: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:53:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:28 smithi100 ceph-mon[39557]: osdmap e394: 8 total, 8 up, 8 in 2024-02-14T17:53:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:28 smithi100 ceph-mon[30138]: pgmap v559: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:53:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:28 smithi100 ceph-mon[30138]: osdmap e394: 8 total, 8 up, 8 in 2024-02-14T17:53:30.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:30 smithi100 ceph-mon[30138]: pgmap v561: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:53:30.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:30 smithi100 ceph-mon[39557]: pgmap v561: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:53:30.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:53:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:53:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:30 smithi179 ceph-mon[33633]: pgmap v561: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:53:32.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:53:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:53:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:32 smithi179 ceph-mon[33633]: pgmap v562: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 225 B/s wr, 1 op/s 2024-02-14T17:53:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:32 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:32 smithi100 ceph-mon[30138]: pgmap v562: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 225 B/s wr, 1 op/s 2024-02-14T17:53:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:32 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:32 smithi100 ceph-mon[39557]: pgmap v562: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 225 B/s wr, 1 op/s 2024-02-14T17:53:33.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:32 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:33 smithi179 ceph-mon[33633]: osdmap e395: 8 total, 8 up, 8 in 2024-02-14T17:53:34.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:33 smithi100 ceph-mon[30138]: osdmap e395: 8 total, 8 up, 8 in 2024-02-14T17:53:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:33 smithi100 ceph-mon[39557]: osdmap e395: 8 total, 8 up, 8 in 2024-02-14T17:53:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:34 smithi179 ceph-mon[33633]: pgmap v564: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:53:35.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:34 smithi100 ceph-mon[30138]: pgmap v564: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:53:35.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:34 smithi100 ceph-mon[39557]: pgmap v564: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:53:37.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:36 smithi179 ceph-mon[33633]: pgmap v565: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T17:53:37.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:36 smithi100 ceph-mon[30138]: pgmap v565: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T17:53:37.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:36 smithi100 ceph-mon[39557]: pgmap v565: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T17:53:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:38 smithi179 ceph-mon[33633]: pgmap v566: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:38 smithi179 ceph-mon[33633]: osdmap e396: 8 total, 8 up, 8 in 2024-02-14T17:53:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:38 smithi100 ceph-mon[39557]: pgmap v566: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:38 smithi100 ceph-mon[39557]: osdmap e396: 8 total, 8 up, 8 in 2024-02-14T17:53:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:38 smithi100 ceph-mon[30138]: pgmap v566: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:38 smithi100 ceph-mon[30138]: osdmap e396: 8 total, 8 up, 8 in 2024-02-14T17:53:40.760 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:53:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:53:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:41.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:40 smithi179 ceph-mon[33633]: pgmap v568: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:41.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:40 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.14", "id": [2, 5]}]: dispatch 2024-02-14T17:53:41.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:40 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.14", "id": [2, 5]}]: dispatch 2024-02-14T17:53:41.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:40 smithi100 ceph-mon[39557]: pgmap v568: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:41.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:40 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.14", "id": [2, 5]}]: dispatch 2024-02-14T17:53:41.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:40 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.14", "id": [2, 5]}]: dispatch 2024-02-14T17:53:41.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:40 smithi100 ceph-mon[30138]: pgmap v568: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:41.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:40 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.14", "id": [2, 5]}]: dispatch 2024-02-14T17:53:41.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:40 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.14", "id": [2, 5]}]: dispatch 2024-02-14T17:53:42.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.14", "id": [2, 5]}]': finished 2024-02-14T17:53:42.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:41 smithi179 ceph-mon[33633]: osdmap e397: 8 total, 8 up, 8 in 2024-02-14T17:53:42.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:41 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:42.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:53:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:53:42.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.14", "id": [2, 5]}]': finished 2024-02-14T17:53:42.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:41 smithi100 ceph-mon[30138]: osdmap e397: 8 total, 8 up, 8 in 2024-02-14T17:53:42.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:41 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:42.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "106.14", "id": [2, 5]}]': finished 2024-02-14T17:53:42.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:41 smithi100 ceph-mon[39557]: osdmap e397: 8 total, 8 up, 8 in 2024-02-14T17:53:42.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:41 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:42 smithi179 ceph-mon[33633]: pgmap v570: 324 pgs: 1 peering, 323 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:42 smithi179 ceph-mon[33633]: osdmap e398: 8 total, 8 up, 8 in 2024-02-14T17:53:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:42 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:43.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:42 smithi100 ceph-mon[30138]: pgmap v570: 324 pgs: 1 peering, 323 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:42 smithi100 ceph-mon[30138]: osdmap e398: 8 total, 8 up, 8 in 2024-02-14T17:53:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:42 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:42 smithi100 ceph-mon[39557]: pgmap v570: 324 pgs: 1 peering, 323 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:53:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:42 smithi100 ceph-mon[39557]: osdmap e398: 8 total, 8 up, 8 in 2024-02-14T17:53:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:42 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:44.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:43 smithi179 ceph-mon[33633]: osdmap e399: 8 total, 8 up, 8 in 2024-02-14T17:53:44.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:43 smithi100 ceph-mon[30138]: osdmap e399: 8 total, 8 up, 8 in 2024-02-14T17:53:44.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:43 smithi100 ceph-mon[39557]: osdmap e399: 8 total, 8 up, 8 in 2024-02-14T17:53:45.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:44 smithi179 ceph-mon[33633]: pgmap v573: 324 pgs: 1 peering, 323 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 957 B/s rd, 0 op/s 2024-02-14T17:53:45.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:44 smithi100 ceph-mon[39557]: pgmap v573: 324 pgs: 1 peering, 323 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 957 B/s rd, 0 op/s 2024-02-14T17:53:45.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:44 smithi100 ceph-mon[30138]: pgmap v573: 324 pgs: 1 peering, 323 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 957 B/s rd, 0 op/s 2024-02-14T17:53:47.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:46 smithi179 ceph-mon[33633]: pgmap v574: 324 pgs: 1 peering, 323 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:53:47.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:46 smithi100 ceph-mon[30138]: pgmap v574: 324 pgs: 1 peering, 323 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:53:47.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:46 smithi100 ceph-mon[39557]: pgmap v574: 324 pgs: 1 peering, 323 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:53:49.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:48 smithi179 ceph-mon[33633]: pgmap v575: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 4 B/s, 1 objects/s recovering 2024-02-14T17:53:49.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:48 smithi179 ceph-mon[33633]: osdmap e400: 8 total, 8 up, 8 in 2024-02-14T17:53:49.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:48 smithi100 ceph-mon[39557]: pgmap v575: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 4 B/s, 1 objects/s recovering 2024-02-14T17:53:49.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:48 smithi100 ceph-mon[39557]: osdmap e400: 8 total, 8 up, 8 in 2024-02-14T17:53:49.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:48 smithi100 ceph-mon[30138]: pgmap v575: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s; 4 B/s, 1 objects/s recovering 2024-02-14T17:53:49.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:48 smithi100 ceph-mon[30138]: osdmap e400: 8 total, 8 up, 8 in 2024-02-14T17:53:50.787 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:53:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:53:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:53:51.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:50 smithi179 ceph-mon[33633]: pgmap v577: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 777 B/s rd, 0 op/s; 3 B/s, 1 objects/s recovering 2024-02-14T17:53:51.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:50 smithi100 ceph-mon[30138]: pgmap v577: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 777 B/s rd, 0 op/s; 3 B/s, 1 objects/s recovering 2024-02-14T17:53:51.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:50 smithi100 ceph-mon[39557]: pgmap v577: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 777 B/s rd, 0 op/s; 3 B/s, 1 objects/s recovering 2024-02-14T17:53:52.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:53:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:53:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:52 smithi179 ceph-mon[33633]: pgmap v578: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3 B/s, 1 objects/s recovering 2024-02-14T17:53:53.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:52 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:53.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:52 smithi100 ceph-mon[30138]: pgmap v578: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3 B/s, 1 objects/s recovering 2024-02-14T17:53:53.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:52 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:53.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:52 smithi100 ceph-mon[39557]: pgmap v578: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3 B/s, 1 objects/s recovering 2024-02-14T17:53:53.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:52 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:53:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:54.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-63"}]: dispatch 2024-02-14T17:53:54.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-63"}]: dispatch 2024-02-14T17:53:54.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:54.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:54.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-63"}]: dispatch 2024-02-14T17:53:54.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-63"}]: dispatch 2024-02-14T17:53:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:53:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-63"}]: dispatch 2024-02-14T17:53:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-63"}]: dispatch 2024-02-14T17:53:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:54 smithi179 ceph-mon[33633]: pgmap v579: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 2 B/s, 1 objects/s recovering 2024-02-14T17:53:55.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:54 smithi179 ceph-mon[33633]: osdmap e401: 8 total, 8 up, 8 in 2024-02-14T17:53:55.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:54 smithi100 ceph-mon[30138]: pgmap v579: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 2 B/s, 1 objects/s recovering 2024-02-14T17:53:55.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:54 smithi100 ceph-mon[30138]: osdmap e401: 8 total, 8 up, 8 in 2024-02-14T17:53:55.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:54 smithi100 ceph-mon[39557]: pgmap v579: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 2 B/s, 1 objects/s recovering 2024-02-14T17:53:55.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:54 smithi100 ceph-mon[39557]: osdmap e401: 8 total, 8 up, 8 in 2024-02-14T17:53:56.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:55 smithi179 ceph-mon[33633]: osdmap e402: 8 total, 8 up, 8 in 2024-02-14T17:53:56.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:56.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:56.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:55 smithi100 ceph-mon[39557]: osdmap e402: 8 total, 8 up, 8 in 2024-02-14T17:53:56.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:56.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:56.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:55 smithi100 ceph-mon[30138]: osdmap e402: 8 total, 8 up, 8 in 2024-02-14T17:53:56.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:56.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:53:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:56 smithi179 ceph-mon[33633]: pgmap v582: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:56 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:56 smithi179 ceph-mon[33633]: osdmap e403: 8 total, 8 up, 8 in 2024-02-14T17:53:57.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:57.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:56 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:57.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[30138]: pgmap v582: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[30138]: osdmap e403: 8 total, 8 up, 8 in 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[39557]: pgmap v582: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:53:57.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[39557]: osdmap e403: 8 total, 8 up, 8 in 2024-02-14T17:53:57.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:53:57.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:57.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:53:57.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:56 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:53:58.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:58.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:57 smithi100 ceph-mon[39557]: osdmap e404: 8 total, 8 up, 8 in 2024-02-14T17:53:58.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:58.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:57 smithi100 ceph-mon[30138]: osdmap e404: 8 total, 8 up, 8 in 2024-02-14T17:53:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:53:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:57 smithi179 ceph-mon[33633]: osdmap e404: 8 total, 8 up, 8 in 2024-02-14T17:53:59.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:58 smithi100 ceph-mon[39557]: pgmap v585: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:59.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:58 smithi100 ceph-mon[39557]: osdmap e405: 8 total, 8 up, 8 in 2024-02-14T17:53:59.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:58 smithi100 ceph-mon[30138]: pgmap v585: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:59.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:58 smithi100 ceph-mon[30138]: osdmap e405: 8 total, 8 up, 8 in 2024-02-14T17:53:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:58 smithi179 ceph-mon[33633]: pgmap v585: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:53:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:58 smithi179 ceph-mon[33633]: osdmap e405: 8 total, 8 up, 8 in 2024-02-14T17:54:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:53:59 smithi100 ceph-mon[30138]: osdmap e406: 8 total, 8 up, 8 in 2024-02-14T17:54:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:53:59 smithi100 ceph-mon[39557]: osdmap e406: 8 total, 8 up, 8 in 2024-02-14T17:54:00.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:53:59 smithi179 ceph-mon[33633]: osdmap e406: 8 total, 8 up, 8 in 2024-02-14T17:54:00.840 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:54:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:54:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:01.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:00 smithi100 ceph-mon[30138]: pgmap v588: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:01.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:00 smithi100 ceph-mon[30138]: osdmap e407: 8 total, 8 up, 8 in 2024-02-14T17:54:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:00 smithi100 ceph-mon[39557]: pgmap v588: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:01.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:00 smithi100 ceph-mon[39557]: osdmap e407: 8 total, 8 up, 8 in 2024-02-14T17:54:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:00 smithi179 ceph-mon[33633]: pgmap v588: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:54:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:00 smithi179 ceph-mon[33633]: osdmap e407: 8 total, 8 up, 8 in 2024-02-14T17:54:02.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:54:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:54:03.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:02 smithi100 ceph-mon[30138]: pgmap v590: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-02-14T17:54:03.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:02 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:02 smithi100 ceph-mon[39557]: pgmap v590: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-02-14T17:54:03.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:02 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:02 smithi179 ceph-mon[33633]: pgmap v590: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 3.1 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-02-14T17:54:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:02 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:04.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:03 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:04.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:03 smithi100 ceph-mon[30138]: osdmap e408: 8 total, 8 up, 8 in 2024-02-14T17:54:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:03 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:04.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:03 smithi100 ceph-mon[39557]: osdmap e408: 8 total, 8 up, 8 in 2024-02-14T17:54:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:03 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:03 smithi179 ceph-mon[33633]: osdmap e408: 8 total, 8 up, 8 in 2024-02-14T17:54:05.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:04 smithi100 ceph-mon[30138]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-14T17:54:05.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:04 smithi100 ceph-mon[39557]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-14T17:54:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:04 smithi179 ceph-mon[33633]: pgmap v592: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.6 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-14T17:54:07.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:06 smithi100 ceph-mon[30138]: pgmap v593: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:54:07.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:06 smithi100 ceph-mon[39557]: pgmap v593: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:54:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:06 smithi179 ceph-mon[33633]: pgmap v593: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-14T17:54:09.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:08 smithi100 ceph-mon[30138]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-14T17:54:09.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:08 smithi100 ceph-mon[39557]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-14T17:54:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:08 smithi179 ceph-mon[33633]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-14T17:54:10.865 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:54:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:11.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-mon[39557]: pgmap v595: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-02-14T17:54:11.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:11.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:11.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-65"}]: dispatch 2024-02-14T17:54:11.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-65"}]: dispatch 2024-02-14T17:54:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-mon[30138]: pgmap v595: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-02-14T17:54:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-65"}]: dispatch 2024-02-14T17:54:11.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-65"}]: dispatch 2024-02-14T17:54:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:10 smithi179 ceph-mon[33633]: pgmap v595: 324 pgs: 324 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-02-14T17:54:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-65"}]: dispatch 2024-02-14T17:54:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-65"}]: dispatch 2024-02-14T17:54:12.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:11 smithi100 ceph-mon[39557]: osdmap e409: 8 total, 8 up, 8 in 2024-02-14T17:54:12.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:12.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:11 smithi100 ceph-mon[30138]: osdmap e409: 8 total, 8 up, 8 in 2024-02-14T17:54:12.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:12.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:54:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:54:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:11 smithi179 ceph-mon[33633]: osdmap e409: 8 total, 8 up, 8 in 2024-02-14T17:54:12.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:13.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:12 smithi100 ceph-mon[30138]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:13.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:12 smithi100 ceph-mon[30138]: osdmap e410: 8 total, 8 up, 8 in 2024-02-14T17:54:13.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:13.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:13.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:12 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:12 smithi100 ceph-mon[39557]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:12 smithi100 ceph-mon[39557]: osdmap e410: 8 total, 8 up, 8 in 2024-02-14T17:54:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:13.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:12 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:12 smithi179 ceph-mon[33633]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:12 smithi179 ceph-mon[33633]: osdmap e410: 8 total, 8 up, 8 in 2024-02-14T17:54:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:12 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:14.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:14.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:13 smithi100 ceph-mon[30138]: osdmap e411: 8 total, 8 up, 8 in 2024-02-14T17:54:14.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:14.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:14.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:13 smithi100 ceph-mon[39557]: osdmap e411: 8 total, 8 up, 8 in 2024-02-14T17:54:14.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:13 smithi179 ceph-mon[33633]: osdmap e411: 8 total, 8 up, 8 in 2024-02-14T17:54:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:15.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[30138]: pgmap v600: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:15.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[30138]: osdmap e412: 8 total, 8 up, 8 in 2024-02-14T17:54:15.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:15.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:15.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-67"}]: dispatch 2024-02-14T17:54:15.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-67"}]: dispatch 2024-02-14T17:54:15.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[39557]: pgmap v600: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:15.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[39557]: osdmap e412: 8 total, 8 up, 8 in 2024-02-14T17:54:15.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:15.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:15.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-67"}]: dispatch 2024-02-14T17:54:15.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-67"}]: dispatch 2024-02-14T17:54:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:14 smithi179 ceph-mon[33633]: pgmap v600: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:14 smithi179 ceph-mon[33633]: osdmap e412: 8 total, 8 up, 8 in 2024-02-14T17:54:15.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:15.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:15.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-67"}]: dispatch 2024-02-14T17:54:15.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-67"}]: dispatch 2024-02-14T17:54:16.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:15 smithi179 ceph-mon[33633]: osdmap e413: 8 total, 8 up, 8 in 2024-02-14T17:54:16.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:15 smithi100 ceph-mon[30138]: osdmap e413: 8 total, 8 up, 8 in 2024-02-14T17:54:16.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:15 smithi100 ceph-mon[39557]: osdmap e413: 8 total, 8 up, 8 in 2024-02-14T17:54:16.497 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477edf0 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477ef40 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477f090 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477f3e0 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477f870 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477f9a0 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477d130 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477d260 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477ff70 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477fad0 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477fc00 2024-02-14T17:54:16.498 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477fd30 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477fe40 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4780540 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47800a0 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47801b0 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4780300 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4780670 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4780b00 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4780c30 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4780430 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47807a0 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47808d0 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4780d60 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47811f0 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4781320 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4780e70 2024-02-14T17:54:16.499 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4780fc0 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47818f0 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4781450 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4781580 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47816b0 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47817e0 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4781ec0 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4781a20 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4781b50 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4781c80 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4781ff0 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47824d0 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782600 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782bd0 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782120 2024-02-14T17:54:16.500 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782250 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782730 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782860 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782380 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782990 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782ac0 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782d00 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4783170 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47832c0 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4783890 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782e10 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4782f60 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47833f0 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4783520 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4783630 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4783760 2024-02-14T17:54:16.501 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4783e80 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47839e0 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4783b10 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4783c40 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4783fb0 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4784440 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4784570 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47840c0 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4784210 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4783d70 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47846a0 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47848c0 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4784eb0 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4784a10 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4784b20 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4784c50 2024-02-14T17:54:16.502 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4784d80 2024-02-14T17:54:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4785480 2024-02-14T17:54:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4784fe0 2024-02-14T17:54:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4785110 2024-02-14T17:54:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4785240 2024-02-14T17:54:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4785370 2024-02-14T17:54:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4785a50 2024-02-14T17:54:16.503 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47855b0 2024-02-14T17:54:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:16 smithi179 ceph-mon[33633]: pgmap v603: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:16 smithi179 ceph-mon[33633]: osdmap e414: 8 total, 8 up, 8 in 2024-02-14T17:54:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:17.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:16 smithi100 ceph-mon[30138]: pgmap v603: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:16 smithi100 ceph-mon[30138]: osdmap e414: 8 total, 8 up, 8 in 2024-02-14T17:54:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:17.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:16 smithi100 ceph-mon[39557]: pgmap v603: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:17.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:16 smithi100 ceph-mon[39557]: osdmap e414: 8 total, 8 up, 8 in 2024-02-14T17:54:17.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:17.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:17 smithi179 ceph-mon[33633]: osdmap e415: 8 total, 8 up, 8 in 2024-02-14T17:54:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-69"}]: dispatch 2024-02-14T17:54:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-69"}]: dispatch 2024-02-14T17:54:18.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[30138]: osdmap e415: 8 total, 8 up, 8 in 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-69"}]: dispatch 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-69"}]: dispatch 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[39557]: osdmap e415: 8 total, 8 up, 8 in 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:18.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-69"}]: dispatch 2024-02-14T17:54:18.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-69"}]: dispatch 2024-02-14T17:54:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:18 smithi179 ceph-mon[33633]: pgmap v606: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:54:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:18 smithi179 ceph-mon[33633]: osdmap e416: 8 total, 8 up, 8 in 2024-02-14T17:54:19.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:18 smithi100 ceph-mon[30138]: pgmap v606: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:54:19.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:18 smithi100 ceph-mon[30138]: osdmap e416: 8 total, 8 up, 8 in 2024-02-14T17:54:19.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:18 smithi100 ceph-mon[39557]: pgmap v606: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:54:19.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:18 smithi100 ceph-mon[39557]: osdmap e416: 8 total, 8 up, 8 in 2024-02-14T17:54:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:19 smithi179 ceph-mon[33633]: osdmap e417: 8 total, 8 up, 8 in 2024-02-14T17:54:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:20.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:19 smithi100 ceph-mon[30138]: osdmap e417: 8 total, 8 up, 8 in 2024-02-14T17:54:20.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:20.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:20.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:19 smithi100 ceph-mon[39557]: osdmap e417: 8 total, 8 up, 8 in 2024-02-14T17:54:20.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:20.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:54:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:20 smithi179 ceph-mon[33633]: pgmap v609: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:54:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:20 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:20 smithi179 ceph-mon[33633]: osdmap e418: 8 total, 8 up, 8 in 2024-02-14T17:54:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:21.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:21.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:21.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[30138]: pgmap v609: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[30138]: osdmap e418: 8 total, 8 up, 8 in 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[39557]: pgmap v609: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[39557]: osdmap e418: 8 total, 8 up, 8 in 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:54:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:54:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:21 smithi179 ceph-mon[33633]: osdmap e419: 8 total, 8 up, 8 in 2024-02-14T17:54:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:21 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:54:22.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:54:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:54:22.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:54:22.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:21 smithi100 ceph-mon[39557]: osdmap e419: 8 total, 8 up, 8 in 2024-02-14T17:54:22.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:21 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:54:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:54:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:21 smithi100 ceph-mon[30138]: osdmap e419: 8 total, 8 up, 8 in 2024-02-14T17:54:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:21 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:54:23.221 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:22 smithi179 ceph-mon[33633]: pgmap v612: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-14T17:54:23.222 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:22 smithi179 ceph-mon[33633]: osdmap e420: 8 total, 8 up, 8 in 2024-02-14T17:54:23.222 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:22 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:23.222 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:22 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:54:23.222 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:22 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:54:23.222 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:22 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:54:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[30138]: pgmap v612: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-14T17:54:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[30138]: osdmap e420: 8 total, 8 up, 8 in 2024-02-14T17:54:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:23.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:54:23.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:54:23.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:54:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[39557]: pgmap v612: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-14T17:54:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[39557]: osdmap e420: 8 total, 8 up, 8 in 2024-02-14T17:54:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:54:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:54:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:22 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:54:25.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:24 smithi179 ceph-mon[33633]: pgmap v614: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-02-14T17:54:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:24 smithi100 ceph-mon[30138]: pgmap v614: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-02-14T17:54:25.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:24 smithi100 ceph-mon[39557]: pgmap v614: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.5 KiB/s wr, 5 op/s 2024-02-14T17:54:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:26 smithi179 ceph-mon[33633]: pgmap v615: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:54:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:26 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:26 smithi100 ceph-mon[30138]: pgmap v615: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:54:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:26 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:27.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:26 smithi100 ceph-mon[39557]: pgmap v615: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-14T17:54:27.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:26 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:29.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:28 smithi179 ceph-mon[33633]: pgmap v616: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-02-14T17:54:29.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:28 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:29.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:28 smithi100 ceph-mon[39557]: pgmap v616: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-02-14T17:54:29.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:28 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:28 smithi100 ceph-mon[30138]: pgmap v616: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-02-14T17:54:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:28 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:54:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:54:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:54:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:30 smithi179 ceph-mon[33633]: pgmap v617: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-14T17:54:31.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:30 smithi100 ceph-mon[30138]: pgmap v617: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-14T17:54:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:30 smithi100 ceph-mon[39557]: pgmap v617: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-14T17:54:32.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:54:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:54:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:32 smithi179 ceph-mon[33633]: pgmap v618: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 409 B/s wr, 1 op/s 2024-02-14T17:54:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:32 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:33.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:32 smithi100 ceph-mon[30138]: pgmap v618: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 409 B/s wr, 1 op/s 2024-02-14T17:54:33.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:32 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:32 smithi100 ceph-mon[39557]: pgmap v618: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 409 B/s wr, 1 op/s 2024-02-14T17:54:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:32 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:34 smithi179 ceph-mon[33633]: pgmap v619: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 959 B/s rd, 348 B/s wr, 1 op/s 2024-02-14T17:54:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:34 smithi100 ceph-mon[30138]: pgmap v619: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 959 B/s rd, 348 B/s wr, 1 op/s 2024-02-14T17:54:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:34 smithi100 ceph-mon[39557]: pgmap v619: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 959 B/s rd, 348 B/s wr, 1 op/s 2024-02-14T17:54:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:36 smithi179 ceph-mon[33633]: pgmap v620: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-02-14T17:54:37.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:36 smithi100 ceph-mon[39557]: pgmap v620: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-02-14T17:54:37.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:36 smithi100 ceph-mon[30138]: pgmap v620: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-02-14T17:54:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:38 smithi179 ceph-mon[33633]: pgmap v621: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-02-14T17:54:39.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:38 smithi100 ceph-mon[30138]: pgmap v621: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-02-14T17:54:39.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:38 smithi100 ceph-mon[39557]: pgmap v621: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-02-14T17:54:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:54:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:54:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:41.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:40 smithi179 ceph-mon[33633]: pgmap v622: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:54:41.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:40 smithi100 ceph-mon[39557]: pgmap v622: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:54:41.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:40 smithi100 ceph-mon[30138]: pgmap v622: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:54:42.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:41 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:42.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:54:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:54:42.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:41 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:41 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:42 smithi179 ceph-mon[33633]: pgmap v623: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:54:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-71"}]: dispatch 2024-02-14T17:54:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-71"}]: dispatch 2024-02-14T17:54:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:42 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[39557]: pgmap v623: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:54:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-71"}]: dispatch 2024-02-14T17:54:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-71"}]: dispatch 2024-02-14T17:54:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[30138]: pgmap v623: 324 pgs: 324 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:54:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-71"}]: dispatch 2024-02-14T17:54:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-71"}]: dispatch 2024-02-14T17:54:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:42 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:44.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:44 smithi179 ceph-mon[33633]: osdmap e421: 8 total, 8 up, 8 in 2024-02-14T17:54:44.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:44 smithi100 ceph-mon[39557]: osdmap e421: 8 total, 8 up, 8 in 2024-02-14T17:54:44.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:44 smithi100 ceph-mon[30138]: osdmap e421: 8 total, 8 up, 8 in 2024-02-14T17:54:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:45 smithi179 ceph-mon[33633]: pgmap v625: 292 pgs: 292 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-02-14T17:54:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:45 smithi179 ceph-mon[33633]: osdmap e422: 8 total, 8 up, 8 in 2024-02-14T17:54:45.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:45.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:45.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:45 smithi100 ceph-mon[39557]: pgmap v625: 292 pgs: 292 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-02-14T17:54:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:45 smithi100 ceph-mon[39557]: osdmap e422: 8 total, 8 up, 8 in 2024-02-14T17:54:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:45 smithi100 ceph-mon[30138]: pgmap v625: 292 pgs: 292 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-02-14T17:54:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:45 smithi100 ceph-mon[30138]: osdmap e422: 8 total, 8 up, 8 in 2024-02-14T17:54:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:54:46.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:46.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:46 smithi179 ceph-mon[33633]: osdmap e423: 8 total, 8 up, 8 in 2024-02-14T17:54:46.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:46.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:46.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:46 smithi100 ceph-mon[39557]: osdmap e423: 8 total, 8 up, 8 in 2024-02-14T17:54:46.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:54:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:46 smithi100 ceph-mon[30138]: osdmap e423: 8 total, 8 up, 8 in 2024-02-14T17:54:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:54:47.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:47 smithi179 ceph-mon[33633]: pgmap v628: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:47.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:47 smithi179 ceph-mon[33633]: osdmap e424: 8 total, 8 up, 8 in 2024-02-14T17:54:47.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:47 smithi100 ceph-mon[30138]: pgmap v628: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:47.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:47 smithi100 ceph-mon[30138]: osdmap e424: 8 total, 8 up, 8 in 2024-02-14T17:54:47.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:47 smithi100 ceph-mon[39557]: pgmap v628: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:54:47.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:47 smithi100 ceph-mon[39557]: osdmap e424: 8 total, 8 up, 8 in 2024-02-14T17:54:48.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:48 smithi179 ceph-mon[33633]: osdmap e425: 8 total, 8 up, 8 in 2024-02-14T17:54:48.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:48 smithi100 ceph-mon[30138]: osdmap e425: 8 total, 8 up, 8 in 2024-02-14T17:54:48.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:48 smithi100 ceph-mon[39557]: osdmap e425: 8 total, 8 up, 8 in 2024-02-14T17:54:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:49 smithi179 ceph-mon[33633]: pgmap v631: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:54:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:49 smithi100 ceph-mon[30138]: pgmap v631: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:54:49.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:49 smithi100 ceph-mon[39557]: pgmap v631: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:54:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:54:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:54:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:54:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:51 smithi179 ceph-mon[33633]: pgmap v632: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 899 B/s rd, 1.1 KiB/s wr, 1 op/s 2024-02-14T17:54:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:51 smithi100 ceph-mon[30138]: pgmap v632: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 899 B/s rd, 1.1 KiB/s wr, 1 op/s 2024-02-14T17:54:51.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:51 smithi100 ceph-mon[39557]: pgmap v632: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 899 B/s rd, 1.1 KiB/s wr, 1 op/s 2024-02-14T17:54:52.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:54:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:54:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:53 smithi179 ceph-mon[33633]: pgmap v633: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-02-14T17:54:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:53 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:53 smithi100 ceph-mon[30138]: pgmap v633: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-02-14T17:54:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:53 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:53 smithi100 ceph-mon[39557]: pgmap v633: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-02-14T17:54:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:53 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:54:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:55 smithi179 ceph-mon[33633]: pgmap v634: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-14T17:54:55.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:55 smithi100 ceph-mon[30138]: pgmap v634: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-14T17:54:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:55 smithi100 ceph-mon[39557]: pgmap v634: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-14T17:54:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:57 smithi179 ceph-mon[33633]: pgmap v635: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 740 B/s wr, 3 op/s 2024-02-14T17:54:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:57 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:57.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:57 smithi100 ceph-mon[39557]: pgmap v635: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 740 B/s wr, 3 op/s 2024-02-14T17:54:57.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:57 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:57 smithi100 ceph-mon[30138]: pgmap v635: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 740 B/s wr, 3 op/s 2024-02-14T17:54:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:57 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:54:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-73"}]: dispatch 2024-02-14T17:54:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-73"}]: dispatch 2024-02-14T17:54:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:58.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-73"}]: dispatch 2024-02-14T17:54:58.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-73"}]: dispatch 2024-02-14T17:54:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:54:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-73"}]: dispatch 2024-02-14T17:54:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-73"}]: dispatch 2024-02-14T17:54:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:59 smithi179 ceph-mon[33633]: pgmap v636: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 95 B/s wr, 2 op/s 2024-02-14T17:54:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:54:59 smithi179 ceph-mon[33633]: osdmap e426: 8 total, 8 up, 8 in 2024-02-14T17:54:59.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:59 smithi100 ceph-mon[30138]: pgmap v636: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 95 B/s wr, 2 op/s 2024-02-14T17:54:59.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:54:59 smithi100 ceph-mon[30138]: osdmap e426: 8 total, 8 up, 8 in 2024-02-14T17:54:59.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:59 smithi100 ceph-mon[39557]: pgmap v636: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 95 B/s wr, 2 op/s 2024-02-14T17:54:59.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:54:59 smithi100 ceph-mon[39557]: osdmap e426: 8 total, 8 up, 8 in 2024-02-14T17:55:00.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:00 smithi179 ceph-mon[33633]: osdmap e427: 8 total, 8 up, 8 in 2024-02-14T17:55:00.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:00.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:00.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:00 smithi100 ceph-mon[30138]: osdmap e427: 8 total, 8 up, 8 in 2024-02-14T17:55:00.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:00.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:00.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:00 smithi100 ceph-mon[39557]: osdmap e427: 8 total, 8 up, 8 in 2024-02-14T17:55:00.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:00.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:55:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:55:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:01 smithi179 ceph-mon[33633]: pgmap v639: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:01 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:01 smithi179 ceph-mon[33633]: osdmap e428: 8 total, 8 up, 8 in 2024-02-14T17:55:01.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:01.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:01 smithi100 ceph-mon[30138]: pgmap v639: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:01 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:01 smithi100 ceph-mon[30138]: osdmap e428: 8 total, 8 up, 8 in 2024-02-14T17:55:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:01 smithi100 ceph-mon[39557]: pgmap v639: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:01 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:01 smithi100 ceph-mon[39557]: osdmap e428: 8 total, 8 up, 8 in 2024-02-14T17:55:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:02.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:02 smithi179 ceph-mon[33633]: osdmap e429: 8 total, 8 up, 8 in 2024-02-14T17:55:02.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:02 smithi179 ceph-mon[33633]: pgmap v642: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:02.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:55:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:55:02.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:02 smithi100 ceph-mon[30138]: osdmap e429: 8 total, 8 up, 8 in 2024-02-14T17:55:02.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:02 smithi100 ceph-mon[30138]: pgmap v642: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:02.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:02 smithi100 ceph-mon[39557]: osdmap e429: 8 total, 8 up, 8 in 2024-02-14T17:55:02.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:02 smithi100 ceph-mon[39557]: pgmap v642: 324 pgs: 324 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:03.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[30138]: osdmap e430: 8 total, 8 up, 8 in 2024-02-14T17:55:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-75"}]: dispatch 2024-02-14T17:55:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-75"}]: dispatch 2024-02-14T17:55:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[39557]: osdmap e430: 8 total, 8 up, 8 in 2024-02-14T17:55:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-75"}]: dispatch 2024-02-14T17:55:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-75"}]: dispatch 2024-02-14T17:55:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:03 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:03 smithi179 ceph-mon[33633]: osdmap e430: 8 total, 8 up, 8 in 2024-02-14T17:55:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-75"}]: dispatch 2024-02-14T17:55:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-75"}]: dispatch 2024-02-14T17:55:04.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:04 smithi100 ceph-mon[39557]: osdmap e431: 8 total, 8 up, 8 in 2024-02-14T17:55:04.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:04 smithi100 ceph-mon[39557]: pgmap v645: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:55:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:04 smithi100 ceph-mon[30138]: osdmap e431: 8 total, 8 up, 8 in 2024-02-14T17:55:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:04 smithi100 ceph-mon[30138]: pgmap v645: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:55:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:04 smithi179 ceph-mon[33633]: osdmap e431: 8 total, 8 up, 8 in 2024-02-14T17:55:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:04 smithi179 ceph-mon[33633]: pgmap v645: 292 pgs: 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:55:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:05 smithi100 ceph-mon[30138]: osdmap e432: 8 total, 8 up, 8 in 2024-02-14T17:55:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:05 smithi100 ceph-mon[39557]: osdmap e432: 8 total, 8 up, 8 in 2024-02-14T17:55:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:05 smithi179 ceph-mon[33633]: osdmap e432: 8 total, 8 up, 8 in 2024-02-14T17:55:05.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:05.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:06.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:06.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:06 smithi100 ceph-mon[39557]: osdmap e433: 8 total, 8 up, 8 in 2024-02-14T17:55:06.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:06.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:06 smithi100 ceph-mon[39557]: pgmap v648: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:06.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:06.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:06 smithi100 ceph-mon[30138]: osdmap e433: 8 total, 8 up, 8 in 2024-02-14T17:55:06.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:06.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:06 smithi100 ceph-mon[30138]: pgmap v648: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:06 smithi179 ceph-mon[33633]: osdmap e433: 8 total, 8 up, 8 in 2024-02-14T17:55:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:06 smithi179 ceph-mon[33633]: pgmap v648: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:07.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:07 smithi100 ceph-mon[39557]: osdmap e434: 8 total, 8 up, 8 in 2024-02-14T17:55:07.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-77"}]: dispatch 2024-02-14T17:55:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-77"}]: dispatch 2024-02-14T17:55:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:07 smithi100 ceph-mon[30138]: osdmap e434: 8 total, 8 up, 8 in 2024-02-14T17:55:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-77"}]: dispatch 2024-02-14T17:55:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-77"}]: dispatch 2024-02-14T17:55:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:07 smithi179 ceph-mon[33633]: osdmap e434: 8 total, 8 up, 8 in 2024-02-14T17:55:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-77"}]: dispatch 2024-02-14T17:55:07.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-77"}]: dispatch 2024-02-14T17:55:08.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:08 smithi100 ceph-mon[39557]: osdmap e435: 8 total, 8 up, 8 in 2024-02-14T17:55:08.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:08 smithi100 ceph-mon[39557]: pgmap v651: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s rd, 767 B/s wr, 142 op/s 2024-02-14T17:55:08.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:08 smithi100 ceph-mon[30138]: osdmap e435: 8 total, 8 up, 8 in 2024-02-14T17:55:08.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:08 smithi100 ceph-mon[30138]: pgmap v651: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s rd, 767 B/s wr, 142 op/s 2024-02-14T17:55:08.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:08 smithi179 ceph-mon[33633]: osdmap e435: 8 total, 8 up, 8 in 2024-02-14T17:55:08.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:08 smithi179 ceph-mon[33633]: pgmap v651: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s rd, 767 B/s wr, 142 op/s 2024-02-14T17:55:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:09 smithi100 ceph-mon[39557]: osdmap e436: 8 total, 8 up, 8 in 2024-02-14T17:55:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:09 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:09 smithi100 ceph-mon[30138]: osdmap e436: 8 total, 8 up, 8 in 2024-02-14T17:55:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:09 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:09 smithi179 ceph-mon[33633]: osdmap e436: 8 total, 8 up, 8 in 2024-02-14T17:55:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:09 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:10.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:10.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:10 smithi100 ceph-mon[30138]: osdmap e437: 8 total, 8 up, 8 in 2024-02-14T17:55:10.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:10.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:10 smithi100 ceph-mon[30138]: pgmap v654: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s rd, 767 B/s wr, 142 op/s 2024-02-14T17:55:10.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:10.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:10 smithi100 ceph-mon[39557]: osdmap e437: 8 total, 8 up, 8 in 2024-02-14T17:55:10.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:10.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:10 smithi100 ceph-mon[39557]: pgmap v654: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s rd, 767 B/s wr, 142 op/s 2024-02-14T17:55:10.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:10.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:10 smithi179 ceph-mon[33633]: osdmap e437: 8 total, 8 up, 8 in 2024-02-14T17:55:10.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:10.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:10 smithi179 ceph-mon[33633]: pgmap v654: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 85 KiB/s rd, 767 B/s wr, 142 op/s 2024-02-14T17:55:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:55:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:55:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:11 smithi179 ceph-mon[33633]: osdmap e438: 8 total, 8 up, 8 in 2024-02-14T17:55:11.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:11 smithi100 ceph-mon[30138]: osdmap e438: 8 total, 8 up, 8 in 2024-02-14T17:55:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:11 smithi100 ceph-mon[39557]: osdmap e438: 8 total, 8 up, 8 in 2024-02-14T17:55:12.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:55:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:55:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:12 smithi179 ceph-mon[33633]: osdmap e439: 8 total, 8 up, 8 in 2024-02-14T17:55:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "303.18"}]: dispatch 2024-02-14T17:55:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:12 smithi179 ceph-mon[33633]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "303.18"}]: dispatch 2024-02-14T17:55:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:12 smithi179 ceph-mon[33633]: 303.18 deep-scrub starts 2024-02-14T17:55:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:12 smithi179 ceph-mon[33633]: 303.18 deep-scrub ok 2024-02-14T17:55:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:12 smithi179 ceph-mon[33633]: pgmap v657: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 1.2 KiB/s wr, 43 op/s 2024-02-14T17:55:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:12 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:12.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[30138]: osdmap e439: 8 total, 8 up, 8 in 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "303.18"}]: dispatch 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[30138]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "303.18"}]: dispatch 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[30138]: 303.18 deep-scrub starts 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[30138]: 303.18 deep-scrub ok 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[30138]: pgmap v657: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 1.2 KiB/s wr, 43 op/s 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[39557]: osdmap e439: 8 total, 8 up, 8 in 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "303.18"}]: dispatch 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[39557]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "303.18"}]: dispatch 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[39557]: 303.18 deep-scrub starts 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[39557]: 303.18 deep-scrub ok 2024-02-14T17:55:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[39557]: pgmap v657: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 25 KiB/s rd, 1.2 KiB/s wr, 43 op/s 2024-02-14T17:55:12.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:12 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:13 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:13 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:13 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:14 smithi179 ceph-mon[33633]: pgmap v658: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 918 B/s wr, 31 op/s 2024-02-14T17:55:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:14 smithi100 ceph-mon[30138]: pgmap v658: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 918 B/s wr, 31 op/s 2024-02-14T17:55:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:14 smithi100 ceph-mon[39557]: pgmap v658: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 918 B/s wr, 31 op/s 2024-02-14T17:55:17.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:16 smithi179 ceph-mon[33633]: pgmap v659: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 779 B/s wr, 26 op/s 2024-02-14T17:55:17.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:16 smithi100 ceph-mon[30138]: pgmap v659: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 779 B/s wr, 26 op/s 2024-02-14T17:55:17.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:16 smithi100 ceph-mon[39557]: pgmap v659: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 779 B/s wr, 26 op/s 2024-02-14T17:55:19.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:18 smithi179 ceph-mon[33633]: pgmap v660: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 639 B/s wr, 22 op/s 2024-02-14T17:55:19.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:18 smithi100 ceph-mon[30138]: pgmap v660: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 639 B/s wr, 22 op/s 2024-02-14T17:55:19.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:18 smithi100 ceph-mon[39557]: pgmap v660: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 639 B/s wr, 22 op/s 2024-02-14T17:55:20.765 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:55:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:55:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:20 smithi179 ceph-mon[33633]: pgmap v661: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 535 B/s wr, 18 op/s 2024-02-14T17:55:21.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:20 smithi100 ceph-mon[30138]: pgmap v661: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 535 B/s wr, 18 op/s 2024-02-14T17:55:21.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:20 smithi100 ceph-mon[39557]: pgmap v661: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 535 B/s wr, 18 op/s 2024-02-14T17:55:22.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:55:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:55:23.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:22 smithi179 ceph-mon[33633]: pgmap v662: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 484 B/s wr, 17 op/s 2024-02-14T17:55:23.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:22 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:55:23.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:22 smithi100 ceph-mon[30138]: pgmap v662: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 484 B/s wr, 17 op/s 2024-02-14T17:55:23.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:22 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:55:23.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:22 smithi100 ceph-mon[39557]: pgmap v662: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 484 B/s wr, 17 op/s 2024-02-14T17:55:23.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:22 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:55:24.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:23 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:24.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:23 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:55:24.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:23 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:55:24.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:23 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:55:24.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:23 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:24.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:23 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:55:24.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:23 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:55:24.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:23 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:55:24.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:23 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:24.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:23 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:55:24.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:23 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:55:24.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:23 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:55:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:24 smithi179 ceph-mon[33633]: pgmap v663: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:55:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:24 smithi100 ceph-mon[30138]: pgmap v663: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:55:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:24 smithi100 ceph-mon[39557]: pgmap v663: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:55:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:27 smithi179 ceph-mon[33633]: pgmap v664: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:55:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:27 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:27 smithi100 ceph-mon[30138]: pgmap v664: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:55:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:27 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:27.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:27 smithi100 ceph-mon[39557]: pgmap v664: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:55:27.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:27 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:29.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:29 smithi179 ceph-mon[33633]: pgmap v665: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:29.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:29 smithi100 ceph-mon[30138]: pgmap v665: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:29.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:29 smithi100 ceph-mon[39557]: pgmap v665: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:55:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:55:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:31 smithi179 ceph-mon[33633]: pgmap v666: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:55:31.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 17:55:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T17:55:31.186+0000 7f59defca700 -1 snap_mapper.add_oid found existing snaps mapped on 303:1bc69e1c:test-rados-api-smithi100-79838-80::foo:2, removing 2024-02-14T17:55:31.322 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 17:55:31 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T17:55:31.186+0000 7f51820cf700 -1 snap_mapper.add_oid found existing snaps mapped on 303:1bc69e1c:test-rados-api-smithi100-79838-80::foo:2, removing 2024-02-14T17:55:31.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:31 smithi100 ceph-mon[30138]: pgmap v666: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:55:31.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:31 smithi100 ceph-mon[39557]: pgmap v666: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T17:55:31.400 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 17:55:31 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0[43959]: 2024-02-14T17:55:31.187+0000 7f81209ea700 -1 snap_mapper.add_oid found existing snaps mapped on 303:1bc69e1c:test-rados-api-smithi100-79838-80::foo:2, removing 2024-02-14T17:55:32.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:55:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:55:32.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:32.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:32.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-79"}]: dispatch 2024-02-14T17:55:32.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-79"}]: dispatch 2024-02-14T17:55:32.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:32.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:32.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-79"}]: dispatch 2024-02-14T17:55:32.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-79"}]: dispatch 2024-02-14T17:55:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-79"}]: dispatch 2024-02-14T17:55:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-79"}]: dispatch 2024-02-14T17:55:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:33 smithi179 ceph-mon[33633]: pgmap v667: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-14T17:55:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:33 smithi179 ceph-mon[33633]: osdmap e440: 8 total, 8 up, 8 in 2024-02-14T17:55:33.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:33 smithi100 ceph-mon[39557]: pgmap v667: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-14T17:55:33.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:33 smithi100 ceph-mon[39557]: osdmap e440: 8 total, 8 up, 8 in 2024-02-14T17:55:33.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:33 smithi100 ceph-mon[30138]: pgmap v667: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-14T17:55:33.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:33 smithi100 ceph-mon[30138]: osdmap e440: 8 total, 8 up, 8 in 2024-02-14T17:55:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:34.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:34 smithi179 ceph-mon[33633]: osdmap e441: 8 total, 8 up, 8 in 2024-02-14T17:55:34.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:34.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:34.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:34 smithi100 ceph-mon[39557]: osdmap e441: 8 total, 8 up, 8 in 2024-02-14T17:55:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:34 smithi100 ceph-mon[30138]: osdmap e441: 8 total, 8 up, 8 in 2024-02-14T17:55:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:34.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:35 smithi179 ceph-mon[33633]: pgmap v670: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-14T17:55:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:35 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:35 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:35 smithi179 ceph-mon[33633]: osdmap e442: 8 total, 8 up, 8 in 2024-02-14T17:55:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:35 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[30138]: pgmap v670: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-14T17:55:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[30138]: osdmap e442: 8 total, 8 up, 8 in 2024-02-14T17:55:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[39557]: pgmap v670: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-14T17:55:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[39557]: osdmap e442: 8 total, 8 up, 8 in 2024-02-14T17:55:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:35.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:35 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:55:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:36 smithi179 ceph-mon[33633]: osdmap e443: 8 total, 8 up, 8 in 2024-02-14T17:55:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:36.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:55:36.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:36 smithi100 ceph-mon[30138]: osdmap e443: 8 total, 8 up, 8 in 2024-02-14T17:55:36.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:36.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:55:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:36 smithi100 ceph-mon[39557]: osdmap e443: 8 total, 8 up, 8 in 2024-02-14T17:55:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:37 smithi179 ceph-mon[33633]: pgmap v673: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:55:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:37 smithi179 ceph-mon[33633]: osdmap e444: 8 total, 8 up, 8 in 2024-02-14T17:55:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:37.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:37 smithi100 ceph-mon[30138]: pgmap v673: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:55:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:37 smithi100 ceph-mon[30138]: osdmap e444: 8 total, 8 up, 8 in 2024-02-14T17:55:37.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:37.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:37 smithi100 ceph-mon[39557]: pgmap v673: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:55:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:37 smithi100 ceph-mon[39557]: osdmap e444: 8 total, 8 up, 8 in 2024-02-14T17:55:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:38.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:55:38.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:38 smithi179 ceph-mon[33633]: osdmap e445: 8 total, 8 up, 8 in 2024-02-14T17:55:38.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:38.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:38.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:55:38.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:38 smithi100 ceph-mon[39557]: osdmap e445: 8 total, 8 up, 8 in 2024-02-14T17:55:38.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:38.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:38.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:55:38.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:38 smithi100 ceph-mon[30138]: osdmap e445: 8 total, 8 up, 8 in 2024-02-14T17:55:38.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:38.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:38.514 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47856e0 2024-02-14T17:55:38.515 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47857f0 2024-02-14T17:55:38.515 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4785940 2024-02-14T17:55:38.515 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4786020 2024-02-14T17:55:38.515 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4785b80 2024-02-14T17:55:38.515 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4785cb0 2024-02-14T17:55:38.515 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4785dc0 2024-02-14T17:55:38.515 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4785f10 2024-02-14T17:55:38.515 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4786610 2024-02-14T17:55:38.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4786170 2024-02-14T17:55:38.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47862c0 2024-02-14T17:55:38.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47863f0 2024-02-14T17:55:38.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4786760 2024-02-14T17:55:38.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4786c40 2024-02-14T17:55:38.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4786d70 2024-02-14T17:55:38.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4786890 2024-02-14T17:55:38.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47869c0 2024-02-14T17:55:38.516 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4786af0 2024-02-14T17:55:38.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4786ea0 2024-02-14T17:55:38.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4787330 2024-02-14T17:55:38.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4787480 2024-02-14T17:55:38.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477f4f0 2024-02-14T17:55:38.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c477f620 2024-02-14T17:55:38.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4787a50 2024-02-14T17:55:38.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47875b0 2024-02-14T17:55:38.517 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47876e0 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4787810 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4787940 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788020 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4787b80 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4787cd0 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4787e00 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788150 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47885e0 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788710 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4786fb0 2024-02-14T17:55:38.518 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47870e0 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788ce0 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788840 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788970 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4787210 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788aa0 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788bd0 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788e10 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788f80 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4789570 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47890d0 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4789200 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4789330 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4789460 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4789b40 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47896a0 2024-02-14T17:55:38.519 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47897d0 2024-02-14T17:55:38.520 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c47898e0 2024-02-14T17:55:38.520 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4789a30 2024-02-14T17:55:38.520 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478a110 2024-02-14T17:55:38.520 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4789c70 2024-02-14T17:55:38.520 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4789da0 2024-02-14T17:55:38.520 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4789ef0 2024-02-14T17:55:38.520 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478a260 2024-02-14T17:55:38.520 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478a6f0 2024-02-14T17:55:38.520 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478a870 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788260 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4788390 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478ae40 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478a9a0 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478aad0 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478ac00 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478ad30 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478b410 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478af70 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478b0c0 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478b1f0 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478b540 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478b9d0 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478bb00 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478a370 2024-02-14T17:55:38.521 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478a4a0 2024-02-14T17:55:38.522 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478c0d0 2024-02-14T17:55:38.522 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478bc30 2024-02-14T17:55:38.522 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478bd60 2024-02-14T17:55:38.522 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478be90 2024-02-14T17:55:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:39 smithi179 ceph-mon[33633]: pgmap v676: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:39 smithi179 ceph-mon[33633]: osdmap e446: 8 total, 8 up, 8 in 2024-02-14T17:55:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:55:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:55:39.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:39 smithi100 ceph-mon[39557]: pgmap v676: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:39.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:39.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:39 smithi100 ceph-mon[39557]: osdmap e446: 8 total, 8 up, 8 in 2024-02-14T17:55:39.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:55:39.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:55:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:39 smithi100 ceph-mon[30138]: pgmap v676: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:39 smithi100 ceph-mon[30138]: osdmap e446: 8 total, 8 up, 8 in 2024-02-14T17:55:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:55:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-14T17:55:40.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-14T17:55:40.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:40 smithi100 ceph-mon[39557]: osdmap e447: 8 total, 8 up, 8 in 2024-02-14T17:55:40.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:55:40.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:55:40.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-14T17:55:40.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:40 smithi100 ceph-mon[30138]: osdmap e447: 8 total, 8 up, 8 in 2024-02-14T17:55:40.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:55:40.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:55:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-14T17:55:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:40 smithi179 ceph-mon[33633]: osdmap e447: 8 total, 8 up, 8 in 2024-02-14T17:55:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:55:40.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-14T17:55:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:55:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:55:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:41 smithi100 ceph-mon[30138]: pgmap v679: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:41.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:41 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-14T17:55:41.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:41 smithi100 ceph-mon[30138]: osdmap e448: 8 total, 8 up, 8 in 2024-02-14T17:55:41.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:41 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:41.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:41 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:41 smithi100 ceph-mon[39557]: pgmap v679: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:41 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-14T17:55:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:41 smithi100 ceph-mon[39557]: osdmap e448: 8 total, 8 up, 8 in 2024-02-14T17:55:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:41 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:41 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:41 smithi179 ceph-mon[33633]: pgmap v679: 324 pgs: 324 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:41 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-14T17:55:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:41 smithi179 ceph-mon[33633]: osdmap e448: 8 total, 8 up, 8 in 2024-02-14T17:55:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:41 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:41.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:41 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:42.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[30138]: osdmap e449: 8 total, 8 up, 8 in 2024-02-14T17:55:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-81"}]: dispatch 2024-02-14T17:55:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-81"}]: dispatch 2024-02-14T17:55:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[30138]: pgmap v682: 324 pgs: 324 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 15 KiB/s wr, 40 op/s 2024-02-14T17:55:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:42.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:42.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[39557]: osdmap e449: 8 total, 8 up, 8 in 2024-02-14T17:55:42.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-81"}]: dispatch 2024-02-14T17:55:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-81"}]: dispatch 2024-02-14T17:55:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[39557]: pgmap v682: 324 pgs: 324 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 15 KiB/s wr, 40 op/s 2024-02-14T17:55:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:42.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:55:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:55:42.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:42.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:42 smithi179 ceph-mon[33633]: osdmap e449: 8 total, 8 up, 8 in 2024-02-14T17:55:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-81"}]: dispatch 2024-02-14T17:55:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-81"}]: dispatch 2024-02-14T17:55:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:42 smithi179 ceph-mon[33633]: pgmap v682: 324 pgs: 324 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 15 KiB/s wr, 40 op/s 2024-02-14T17:55:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:43 smithi100 ceph-mon[39557]: osdmap e450: 8 total, 8 up, 8 in 2024-02-14T17:55:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:43 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:43 smithi100 ceph-mon[30138]: osdmap e450: 8 total, 8 up, 8 in 2024-02-14T17:55:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:43 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:43 smithi179 ceph-mon[33633]: osdmap e450: 8 total, 8 up, 8 in 2024-02-14T17:55:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:43 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:44 smithi100 ceph-mon[30138]: osdmap e451: 8 total, 8 up, 8 in 2024-02-14T17:55:44.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:44.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:44.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:44 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:44.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:44 smithi100 ceph-mon[30138]: pgmap v685: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-14T17:55:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:44 smithi100 ceph-mon[39557]: osdmap e451: 8 total, 8 up, 8 in 2024-02-14T17:55:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:44 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:44 smithi100 ceph-mon[39557]: pgmap v685: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-14T17:55:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:44 smithi179 ceph-mon[33633]: osdmap e451: 8 total, 8 up, 8 in 2024-02-14T17:55:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:44 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:55:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:44 smithi179 ceph-mon[33633]: pgmap v685: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-14T17:55:45.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:45.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:45 smithi100 ceph-mon[30138]: osdmap e452: 8 total, 8 up, 8 in 2024-02-14T17:55:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:45 smithi100 ceph-mon[39557]: osdmap e452: 8 total, 8 up, 8 in 2024-02-14T17:55:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:45 smithi179 ceph-mon[33633]: osdmap e452: 8 total, 8 up, 8 in 2024-02-14T17:55:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:55:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[30138]: osdmap e453: 8 total, 8 up, 8 in 2024-02-14T17:55:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[30138]: pgmap v688: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:46.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:55:46.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[30138]: osdmap e454: 8 total, 8 up, 8 in 2024-02-14T17:55:46.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:55:46.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[39557]: osdmap e453: 8 total, 8 up, 8 in 2024-02-14T17:55:46.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:46.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:46.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[39557]: pgmap v688: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:46.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:55:46.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:46 smithi100 ceph-mon[39557]: osdmap e454: 8 total, 8 up, 8 in 2024-02-14T17:55:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:55:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:46 smithi179 ceph-mon[33633]: osdmap e453: 8 total, 8 up, 8 in 2024-02-14T17:55:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:46.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:46 smithi179 ceph-mon[33633]: pgmap v688: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:46.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:55:46.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:46 smithi179 ceph-mon[33633]: osdmap e454: 8 total, 8 up, 8 in 2024-02-14T17:55:47.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:47.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:47.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:55:47.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:47.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:47.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:55:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:55:48.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:48 smithi100 ceph-mon[39557]: osdmap e455: 8 total, 8 up, 8 in 2024-02-14T17:55:48.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:48.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:48.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:48 smithi100 ceph-mon[39557]: pgmap v691: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:48.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:48 smithi100 ceph-mon[30138]: osdmap e455: 8 total, 8 up, 8 in 2024-02-14T17:55:48.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:48.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:48.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:48 smithi100 ceph-mon[30138]: pgmap v691: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:48 smithi179 ceph-mon[33633]: osdmap e455: 8 total, 8 up, 8 in 2024-02-14T17:55:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:48 smithi179 ceph-mon[33633]: pgmap v691: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:49.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:49 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:49.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:49 smithi100 ceph-mon[39557]: osdmap e456: 8 total, 8 up, 8 in 2024-02-14T17:55:49.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:49 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:49.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:49 smithi100 ceph-mon[30138]: osdmap e456: 8 total, 8 up, 8 in 2024-02-14T17:55:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:49 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:49 smithi179 ceph-mon[33633]: osdmap e456: 8 total, 8 up, 8 in 2024-02-14T17:55:50.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:50 smithi100 ceph-mon[39557]: osdmap e457: 8 total, 8 up, 8 in 2024-02-14T17:55:50.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:50 smithi100 ceph-mon[39557]: pgmap v694: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:50 smithi100 ceph-mon[30138]: osdmap e457: 8 total, 8 up, 8 in 2024-02-14T17:55:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:50 smithi100 ceph-mon[30138]: pgmap v694: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:50 smithi179 ceph-mon[33633]: osdmap e457: 8 total, 8 up, 8 in 2024-02-14T17:55:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:50 smithi179 ceph-mon[33633]: pgmap v694: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:55:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:55:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:55:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:51 smithi179 ceph-mon[33633]: osdmap e458: 8 total, 8 up, 8 in 2024-02-14T17:55:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-83"}]: dispatch 2024-02-14T17:55:51.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-83"}]: dispatch 2024-02-14T17:55:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:51 smithi100 ceph-mon[39557]: osdmap e458: 8 total, 8 up, 8 in 2024-02-14T17:55:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-83"}]: dispatch 2024-02-14T17:55:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-83"}]: dispatch 2024-02-14T17:55:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:51 smithi100 ceph-mon[30138]: osdmap e458: 8 total, 8 up, 8 in 2024-02-14T17:55:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-83"}]: dispatch 2024-02-14T17:55:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-83"}]: dispatch 2024-02-14T17:55:52.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:55:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:55:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:52 smithi179 ceph-mon[33633]: osdmap e459: 8 total, 8 up, 8 in 2024-02-14T17:55:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:52 smithi179 ceph-mon[33633]: pgmap v697: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-14T17:55:52.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:52 smithi100 ceph-mon[39557]: osdmap e459: 8 total, 8 up, 8 in 2024-02-14T17:55:52.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:52 smithi100 ceph-mon[39557]: pgmap v697: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-14T17:55:52.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:52 smithi100 ceph-mon[30138]: osdmap e459: 8 total, 8 up, 8 in 2024-02-14T17:55:52.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:52 smithi100 ceph-mon[30138]: pgmap v697: 292 pgs: 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-14T17:55:53.472 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:53 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:53.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:53 smithi179 ceph-mon[33633]: osdmap e460: 8 total, 8 up, 8 in 2024-02-14T17:55:53.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:53.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:53.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:53 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:53.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:53 smithi100 ceph-mon[39557]: osdmap e460: 8 total, 8 up, 8 in 2024-02-14T17:55:53.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:53 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:55:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:53 smithi100 ceph-mon[30138]: osdmap e460: 8 total, 8 up, 8 in 2024-02-14T17:55:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:55:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:54 smithi179 ceph-mon[33633]: osdmap e461: 8 total, 8 up, 8 in 2024-02-14T17:55:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:54 smithi179 ceph-mon[33633]: pgmap v700: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-14T17:55:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[39557]: osdmap e461: 8 total, 8 up, 8 in 2024-02-14T17:55:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[39557]: pgmap v700: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-14T17:55:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:55:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[30138]: osdmap e461: 8 total, 8 up, 8 in 2024-02-14T17:55:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:55:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:55:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:54 smithi100 ceph-mon[30138]: pgmap v700: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-02-14T17:55:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:55:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:55 smithi179 ceph-mon[33633]: osdmap e462: 8 total, 8 up, 8 in 2024-02-14T17:55:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:55 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:55:55.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:55 smithi100 ceph-mon[30138]: osdmap e462: 8 total, 8 up, 8 in 2024-02-14T17:55:55.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:55.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:55 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:55:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:55 smithi100 ceph-mon[39557]: osdmap e462: 8 total, 8 up, 8 in 2024-02-14T17:55:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:55 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:55:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:55:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:56 smithi179 ceph-mon[33633]: osdmap e463: 8 total, 8 up, 8 in 2024-02-14T17:55:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:56 smithi179 ceph-mon[33633]: pgmap v703: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:55:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:56 smithi179 ceph-mon[33633]: osdmap e464: 8 total, 8 up, 8 in 2024-02-14T17:55:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:56.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:55:56.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[30138]: osdmap e463: 8 total, 8 up, 8 in 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[30138]: pgmap v703: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[30138]: osdmap e464: 8 total, 8 up, 8 in 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_tier","val": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[39557]: osdmap e463: 8 total, 8 up, 8 in 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[39557]: pgmap v703: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:55:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:55:56.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[39557]: osdmap e464: 8 total, 8 up, 8 in 2024-02-14T17:55:56.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:57 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:57.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:57 smithi179 ceph-mon[33633]: osdmap e465: 8 total, 8 up, 8 in 2024-02-14T17:55:57.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:57.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:57 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:57 smithi100 ceph-mon[39557]: osdmap e465: 8 total, 8 up, 8 in 2024-02-14T17:55:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:55:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:57 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:55:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:55:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:57 smithi100 ceph-mon[30138]: osdmap e465: 8 total, 8 up, 8 in 2024-02-14T17:55:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:58 smithi179 ceph-mon[33633]: pgmap v706: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:58 smithi100 ceph-mon[39557]: pgmap v706: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:58 smithi100 ceph-mon[30138]: pgmap v706: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:55:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:55:59 smithi179 ceph-mon[33633]: osdmap e466: 8 total, 8 up, 8 in 2024-02-14T17:55:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:55:59 smithi100 ceph-mon[30138]: osdmap e466: 8 total, 8 up, 8 in 2024-02-14T17:55:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:55:59 smithi100 ceph-mon[39557]: osdmap e466: 8 total, 8 up, 8 in 2024-02-14T17:56:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[39557]: osdmap e467: 8 total, 8 up, 8 in 2024-02-14T17:56:00.508 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:00.508 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:00.508 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-85"}]: dispatch 2024-02-14T17:56:00.508 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-85"}]: dispatch 2024-02-14T17:56:00.508 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[39557]: pgmap v709: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:00.508 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[30138]: osdmap e467: 8 total, 8 up, 8 in 2024-02-14T17:56:00.509 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:00.509 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:00.509 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-85"}]: dispatch 2024-02-14T17:56:00.509 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-85"}]: dispatch 2024-02-14T17:56:00.509 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-mon[30138]: pgmap v709: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:00 smithi179 ceph-mon[33633]: osdmap e467: 8 total, 8 up, 8 in 2024-02-14T17:56:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-85"}]: dispatch 2024-02-14T17:56:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-85"}]: dispatch 2024-02-14T17:56:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:00 smithi179 ceph-mon[33633]: pgmap v709: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:56:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:56:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:01 smithi179 ceph-mon[33633]: osdmap e468: 8 total, 8 up, 8 in 2024-02-14T17:56:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:01 smithi100 ceph-mon[30138]: osdmap e468: 8 total, 8 up, 8 in 2024-02-14T17:56:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:01 smithi100 ceph-mon[39557]: osdmap e468: 8 total, 8 up, 8 in 2024-02-14T17:56:02.248 INFO:tasks.workunit.client.0.smithi100.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18069 ms) 2024-02-14T17:56:02.248 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-14T17:56:02.248 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22704 ms) 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3006 ms) 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3021 ms) 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3018 ms) 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3026 ms) 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4030 ms) 2024-02-14T17:56:02.249 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-14T17:56:02.250 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.250 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38285 ms) 2024-02-14T17:56:02.250 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-14T17:56:02.250 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.250 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17060 ms) 2024-02-14T17:56:02.250 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-14T17:56:02.250 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.250 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4036 ms) 2024-02-14T17:56:02.250 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-14T17:56:02.250 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3029 ms) 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25064 ms) 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15058 ms) 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5041 ms) 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4036 ms) 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-14T17:56:02.251 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: waiting for scrubs... 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: done waiting 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24898 ms) 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10082 ms) 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9056 ms) 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9063 ms) 2024-02-14T17:56:02.252 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-14T17:56:02.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:56:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:56:02.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:02 smithi179 ceph-mon[33633]: osdmap e469: 8 total, 8 up, 8 in 2024-02-14T17:56:02.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:02 smithi179 ceph-mon[33633]: pgmap v712: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-14T17:56:02.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:02 smithi100 ceph-mon[30138]: osdmap e469: 8 total, 8 up, 8 in 2024-02-14T17:56:02.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:02 smithi100 ceph-mon[30138]: pgmap v712: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-14T17:56:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:02 smithi100 ceph-mon[39557]: osdmap e469: 8 total, 8 up, 8 in 2024-02-14T17:56:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:02 smithi100 ceph-mon[39557]: pgmap v712: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-14T17:56:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:03 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:03 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:03 smithi179 ceph-mon[33633]: osdmap e470: 8 total, 8 up, 8 in 2024-02-14T17:56:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:03 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:03 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:03 smithi100 ceph-mon[30138]: osdmap e470: 8 total, 8 up, 8 in 2024-02-14T17:56:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:03 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:03 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:03 smithi100 ceph-mon[39557]: osdmap e470: 8 total, 8 up, 8 in 2024-02-14T17:56:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:04 smithi179 ceph-mon[33633]: osdmap e471: 8 total, 8 up, 8 in 2024-02-14T17:56:04.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:04.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:04.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:04 smithi179 ceph-mon[33633]: pgmap v715: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-14T17:56:04.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:56:04.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:04 smithi179 ceph-mon[33633]: osdmap e472: 8 total, 8 up, 8 in 2024-02-14T17:56:04.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_tier","val": "test-rados-api-smithi100-79838-89-test-flush"}]: dispatch 2024-02-14T17:56:04.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_tier","val": "test-rados-api-smithi100-79838-89-test-flush"}]: dispatch 2024-02-14T17:56:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[39557]: osdmap e471: 8 total, 8 up, 8 in 2024-02-14T17:56:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[39557]: pgmap v715: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[39557]: osdmap e472: 8 total, 8 up, 8 in 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_tier","val": "test-rados-api-smithi100-79838-89-test-flush"}]: dispatch 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_tier","val": "test-rados-api-smithi100-79838-89-test-flush"}]: dispatch 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[30138]: osdmap e471: 8 total, 8 up, 8 in 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[30138]: pgmap v715: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-14T17:56:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:56:04.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[30138]: osdmap e472: 8 total, 8 up, 8 in 2024-02-14T17:56:04.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_tier","val": "test-rados-api-smithi100-79838-89-test-flush"}]: dispatch 2024-02-14T17:56:04.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_tier","val": "test-rados-api-smithi100-79838-89-test-flush"}]: dispatch 2024-02-14T17:56:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_tier","val": "test-rados-api-smithi100-79838-89-test-flush"}]': finished 2024-02-14T17:56:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:06 smithi179 ceph-mon[33633]: osdmap e473: 8 total, 8 up, 8 in 2024-02-14T17:56:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:56:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:56:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:06 smithi179 ceph-mon[33633]: pgmap v718: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_tier","val": "test-rados-api-smithi100-79838-89-test-flush"}]': finished 2024-02-14T17:56:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:06 smithi100 ceph-mon[39557]: osdmap e473: 8 total, 8 up, 8 in 2024-02-14T17:56:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:56:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:56:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:06 smithi100 ceph-mon[39557]: pgmap v718: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:06.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_tier","val": "test-rados-api-smithi100-79838-89-test-flush"}]': finished 2024-02-14T17:56:06.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:06 smithi100 ceph-mon[30138]: osdmap e473: 8 total, 8 up, 8 in 2024-02-14T17:56:06.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:56:06.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:56:06.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:06 smithi100 ceph-mon[30138]: pgmap v718: 356 pgs: 32 unknown, 32 creating+peering, 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:56:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:07 smithi179 ceph-mon[33633]: osdmap e474: 8 total, 8 up, 8 in 2024-02-14T17:56:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:56:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:56:07.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:56:07.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:07 smithi100 ceph-mon[39557]: osdmap e474: 8 total, 8 up, 8 in 2024-02-14T17:56:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:56:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:56:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:56:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:07 smithi100 ceph-mon[30138]: osdmap e474: 8 total, 8 up, 8 in 2024-02-14T17:56:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:56:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:56:08.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:56:08.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:08 smithi179 ceph-mon[33633]: osdmap e475: 8 total, 8 up, 8 in 2024-02-14T17:56:08.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:08 smithi179 ceph-mon[33633]: pgmap v721: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:08.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:56:08.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:08 smithi100 ceph-mon[39557]: osdmap e475: 8 total, 8 up, 8 in 2024-02-14T17:56:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:08 smithi100 ceph-mon[39557]: pgmap v721: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:08.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:56:08.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:08 smithi100 ceph-mon[30138]: osdmap e475: 8 total, 8 up, 8 in 2024-02-14T17:56:08.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:08 smithi100 ceph-mon[30138]: pgmap v721: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:09 smithi179 ceph-mon[33633]: osdmap e476: 8 total, 8 up, 8 in 2024-02-14T17:56:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-87"}]: dispatch 2024-02-14T17:56:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-87"}]: dispatch 2024-02-14T17:56:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:09 smithi100 ceph-mon[39557]: osdmap e476: 8 total, 8 up, 8 in 2024-02-14T17:56:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-87"}]: dispatch 2024-02-14T17:56:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-87"}]: dispatch 2024-02-14T17:56:09.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:09 smithi100 ceph-mon[30138]: osdmap e476: 8 total, 8 up, 8 in 2024-02-14T17:56:09.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:09.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:09.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-87"}]: dispatch 2024-02-14T17:56:09.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-87"}]: dispatch 2024-02-14T17:56:10.551 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:56:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:56:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:10.551 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:10 smithi100 ceph-mon[39557]: osdmap e477: 8 total, 8 up, 8 in 2024-02-14T17:56:10.551 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:10 smithi100 ceph-mon[39557]: pgmap v724: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:10.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:10 smithi100 ceph-mon[30138]: osdmap e477: 8 total, 8 up, 8 in 2024-02-14T17:56:10.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:10 smithi100 ceph-mon[30138]: pgmap v724: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:10.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:10 smithi179 ceph-mon[33633]: osdmap e477: 8 total, 8 up, 8 in 2024-02-14T17:56:10.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:10 smithi179 ceph-mon[33633]: pgmap v724: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:56:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:11 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:11 smithi179 ceph-mon[33633]: osdmap e478: 8 total, 8 up, 8 in 2024-02-14T17:56:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:11.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:11.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:11 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:11.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:11 smithi100 ceph-mon[30138]: osdmap e478: 8 total, 8 up, 8 in 2024-02-14T17:56:11.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:11.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:11 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:11 smithi100 ceph-mon[39557]: osdmap e478: 8 total, 8 up, 8 in 2024-02-14T17:56:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:12.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:56:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:56:12.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:12 smithi179 ceph-mon[33633]: osdmap e479: 8 total, 8 up, 8 in 2024-02-14T17:56:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:12 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:12 smithi179 ceph-mon[33633]: pgmap v727: 324 pgs: 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:56:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:12 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:12.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[30138]: osdmap e479: 8 total, 8 up, 8 in 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[30138]: pgmap v727: 324 pgs: 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[39557]: osdmap e479: 8 total, 8 up, 8 in 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:56:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[39557]: pgmap v727: 324 pgs: 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:56:12.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:12 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:13 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:56:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:13 smithi179 ceph-mon[33633]: osdmap e480: 8 total, 8 up, 8 in 2024-02-14T17:56:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:13 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:56:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:13 smithi100 ceph-mon[30138]: osdmap e480: 8 total, 8 up, 8 in 2024-02-14T17:56:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:13 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:56:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:13 smithi100 ceph-mon[39557]: osdmap e480: 8 total, 8 up, 8 in 2024-02-14T17:56:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:14 smithi179 ceph-mon[33633]: osdmap e481: 8 total, 8 up, 8 in 2024-02-14T17:56:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:14 smithi179 ceph-mon[33633]: pgmap v730: 324 pgs: 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:56:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:14 smithi100 ceph-mon[30138]: osdmap e481: 8 total, 8 up, 8 in 2024-02-14T17:56:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:14 smithi100 ceph-mon[30138]: pgmap v730: 324 pgs: 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:56:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:14 smithi100 ceph-mon[39557]: osdmap e481: 8 total, 8 up, 8 in 2024-02-14T17:56:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:14 smithi100 ceph-mon[39557]: pgmap v730: 324 pgs: 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-14T17:56:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:15 smithi179 ceph-mon[33633]: osdmap e482: 8 total, 8 up, 8 in 2024-02-14T17:56:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-90"}]: dispatch 2024-02-14T17:56:15.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-90"}]: dispatch 2024-02-14T17:56:15.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:15 smithi100 ceph-mon[39557]: osdmap e482: 8 total, 8 up, 8 in 2024-02-14T17:56:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-90"}]: dispatch 2024-02-14T17:56:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-90"}]: dispatch 2024-02-14T17:56:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:15 smithi100 ceph-mon[30138]: osdmap e482: 8 total, 8 up, 8 in 2024-02-14T17:56:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:15.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:15.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-90"}]: dispatch 2024-02-14T17:56:15.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-90"}]: dispatch 2024-02-14T17:56:16.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:16 smithi179 ceph-mon[33633]: osdmap e483: 8 total, 8 up, 8 in 2024-02-14T17:56:16.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:16 smithi179 ceph-mon[33633]: pgmap v733: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:16.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:16 smithi100 ceph-mon[30138]: osdmap e483: 8 total, 8 up, 8 in 2024-02-14T17:56:16.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:16 smithi100 ceph-mon[30138]: pgmap v733: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:16.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:16 smithi100 ceph-mon[39557]: osdmap e483: 8 total, 8 up, 8 in 2024-02-14T17:56:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:16 smithi100 ceph-mon[39557]: pgmap v733: 292 pgs: 292 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:17 smithi179 ceph-mon[33633]: osdmap e484: 8 total, 8 up, 8 in 2024-02-14T17:56:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:17 smithi100 ceph-mon[30138]: osdmap e484: 8 total, 8 up, 8 in 2024-02-14T17:56:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:17 smithi100 ceph-mon[39557]: osdmap e484: 8 total, 8 up, 8 in 2024-02-14T17:56:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:18.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:18 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:18.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:18 smithi100 ceph-mon[39557]: osdmap e485: 8 total, 8 up, 8 in 2024-02-14T17:56:18.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:18.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:18 smithi100 ceph-mon[39557]: pgmap v736: 324 pgs: 12 creating+peering, 20 unknown, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:56:18.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:18 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:18.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:18 smithi100 ceph-mon[30138]: osdmap e485: 8 total, 8 up, 8 in 2024-02-14T17:56:18.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:18.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:18 smithi100 ceph-mon[30138]: pgmap v736: 324 pgs: 12 creating+peering, 20 unknown, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:56:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:18 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:18 smithi179 ceph-mon[33633]: osdmap e485: 8 total, 8 up, 8 in 2024-02-14T17:56:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:18.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:18 smithi179 ceph-mon[33633]: pgmap v736: 324 pgs: 12 creating+peering, 20 unknown, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:56:19.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:19 smithi100 ceph-mon[30138]: osdmap e486: 8 total, 8 up, 8 in 2024-02-14T17:56:19.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:19 smithi100 ceph-mon[39557]: osdmap e486: 8 total, 8 up, 8 in 2024-02-14T17:56:19.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:19 smithi179 ceph-mon[33633]: osdmap e486: 8 total, 8 up, 8 in 2024-02-14T17:56:20.651 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:56:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:20.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[39557]: osdmap e487: 8 total, 8 up, 8 in 2024-02-14T17:56:20.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:20.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:20.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-92"}]: dispatch 2024-02-14T17:56:20.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-92"}]: dispatch 2024-02-14T17:56:20.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[39557]: pgmap v739: 324 pgs: 12 creating+peering, 20 unknown, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:56:20.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[30138]: osdmap e487: 8 total, 8 up, 8 in 2024-02-14T17:56:20.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:20.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:20.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-92"}]: dispatch 2024-02-14T17:56:20.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-92"}]: dispatch 2024-02-14T17:56:20.653 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:20 smithi100 ceph-mon[30138]: pgmap v739: 324 pgs: 12 creating+peering, 20 unknown, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:56:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:20 smithi179 ceph-mon[33633]: osdmap e487: 8 total, 8 up, 8 in 2024-02-14T17:56:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-92"}]: dispatch 2024-02-14T17:56:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-92"}]: dispatch 2024-02-14T17:56:20.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:20 smithi179 ceph-mon[33633]: pgmap v739: 324 pgs: 12 creating+peering, 20 unknown, 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-14T17:56:21.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:21 smithi100 ceph-mon[30138]: osdmap e488: 8 total, 8 up, 8 in 2024-02-14T17:56:21.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:21 smithi100 ceph-mon[39557]: osdmap e488: 8 total, 8 up, 8 in 2024-02-14T17:56:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:21 smithi179 ceph-mon[33633]: osdmap e488: 8 total, 8 up, 8 in 2024-02-14T17:56:22.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:56:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:56:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:22 smithi179 ceph-mon[33633]: osdmap e489: 8 total, 8 up, 8 in 2024-02-14T17:56:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:22 smithi179 ceph-mon[33633]: pgmap v742: 324 pgs: 3 creating+peering, 29 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:56:22.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:22 smithi100 ceph-mon[39557]: osdmap e489: 8 total, 8 up, 8 in 2024-02-14T17:56:22.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:22.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:22.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:22 smithi100 ceph-mon[39557]: pgmap v742: 324 pgs: 3 creating+peering, 29 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:56:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:22 smithi100 ceph-mon[30138]: osdmap e489: 8 total, 8 up, 8 in 2024-02-14T17:56:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:22 smithi100 ceph-mon[30138]: pgmap v742: 324 pgs: 3 creating+peering, 29 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:56:23.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:23.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:23.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[39557]: osdmap e490: 8 total, 8 up, 8 in 2024-02-14T17:56:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:56:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[30138]: osdmap e490: 8 total, 8 up, 8 in 2024-02-14T17:56:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:23 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:56:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:23 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:23 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:23 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:23 smithi179 ceph-mon[33633]: osdmap e490: 8 total, 8 up, 8 in 2024-02-14T17:56:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:23.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:23 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:56:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: osdmap e491: 8 total, 8 up, 8 in 2024-02-14T17:56:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi179 to 3248M 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: Adjusting osd_memory_target on smithi100 to 3248M 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[30138]: pgmap v745: 324 pgs: 3 creating+peering, 29 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: osdmap e491: 8 total, 8 up, 8 in 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi179 to 3248M 2024-02-14T17:56:24.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: Adjusting osd_memory_target on smithi100 to 3248M 2024-02-14T17:56:24.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:56:24.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:56:24.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:56:24.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:56:24.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:56:24.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:24 smithi100 ceph-mon[39557]: pgmap v745: 324 pgs: 3 creating+peering, 29 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:56:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: osdmap e491: 8 total, 8 up, 8 in 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi179 to 3248M 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: Adjusting osd_memory_target on smithi100 to 3248M 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:56:24.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:24 smithi179 ceph-mon[33633]: pgmap v745: 324 pgs: 3 creating+peering, 29 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-14T17:56:25.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:25 smithi100 ceph-mon[30138]: osdmap e492: 8 total, 8 up, 8 in 2024-02-14T17:56:25.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:25 smithi100 ceph-mon[39557]: osdmap e492: 8 total, 8 up, 8 in 2024-02-14T17:56:25.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:25 smithi179 ceph-mon[33633]: osdmap e492: 8 total, 8 up, 8 in 2024-02-14T17:56:26.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:26 smithi100 ceph-mon[30138]: pgmap v747: 324 pgs: 3 creating+peering, 29 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:26.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:26 smithi100 ceph-mon[39557]: pgmap v747: 324 pgs: 3 creating+peering, 29 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:26 smithi179 ceph-mon[33633]: pgmap v747: 324 pgs: 3 creating+peering, 29 unknown, 292 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:27.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:27 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:27 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:27.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:27 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:28 smithi100 ceph-mon[30138]: pgmap v748: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-02-14T17:56:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:28 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:28 smithi100 ceph-mon[39557]: pgmap v748: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-02-14T17:56:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:28 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:28.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:28 smithi179 ceph-mon[33633]: pgmap v748: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-02-14T17:56:28.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:28 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:30.786 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:56:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:56:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:30 smithi179 ceph-mon[33633]: pgmap v749: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-14T17:56:31.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:30 smithi100 ceph-mon[30138]: pgmap v749: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-14T17:56:31.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:30 smithi100 ceph-mon[39557]: pgmap v749: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-14T17:56:32.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:56:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:56:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:32 smithi179 ceph-mon[33633]: pgmap v750: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 979 B/s wr, 4 op/s 2024-02-14T17:56:33.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:32 smithi100 ceph-mon[30138]: pgmap v750: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 979 B/s wr, 4 op/s 2024-02-14T17:56:33.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:32 smithi100 ceph-mon[39557]: pgmap v750: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 979 B/s wr, 4 op/s 2024-02-14T17:56:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:33 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:34.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:33 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:34.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:33 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:34 smithi179 ceph-mon[33633]: pgmap v751: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-14T17:56:35.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:34 smithi100 ceph-mon[30138]: pgmap v751: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-14T17:56:35.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:34 smithi100 ceph-mon[39557]: pgmap v751: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-14T17:56:36.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:35 smithi179 ceph-mon[33633]: osdmap e493: 8 total, 8 up, 8 in 2024-02-14T17:56:36.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:35 smithi100 ceph-mon[30138]: osdmap e493: 8 total, 8 up, 8 in 2024-02-14T17:56:36.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:35 smithi100 ceph-mon[39557]: osdmap e493: 8 total, 8 up, 8 in 2024-02-14T17:56:37.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:36 smithi179 ceph-mon[33633]: pgmap v753: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-14T17:56:37.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:36 smithi100 ceph-mon[30138]: pgmap v753: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-14T17:56:37.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:36 smithi100 ceph-mon[39557]: pgmap v753: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-02-14T17:56:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:38 smithi179 ceph-mon[33633]: pgmap v754: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:38 smithi179 ceph-mon[33633]: osdmap e494: 8 total, 8 up, 8 in 2024-02-14T17:56:39.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:38 smithi100 ceph-mon[30138]: pgmap v754: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:39.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:38 smithi100 ceph-mon[30138]: osdmap e494: 8 total, 8 up, 8 in 2024-02-14T17:56:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:38 smithi100 ceph-mon[39557]: pgmap v754: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:38 smithi100 ceph-mon[39557]: osdmap e494: 8 total, 8 up, 8 in 2024-02-14T17:56:40.814 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:56:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:56:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:41.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:40 smithi179 ceph-mon[33633]: pgmap v756: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:41.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:40 smithi100 ceph-mon[30138]: pgmap v756: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:41.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:40 smithi100 ceph-mon[39557]: pgmap v756: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:41.821 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:56:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=cleanup t=2024-02-14T17:56:41.458926271Z level=info msg="Completed cleanup jobs" duration=3.339483ms 2024-02-14T17:56:41.821 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 17:56:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=sqlstore.transactions t=2024-02-14T17:56:41.466152589Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-14T17:56:42.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:56:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:56:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:42 smithi179 ceph-mon[33633]: pgmap v757: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:43.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:42 smithi100 ceph-mon[30138]: pgmap v757: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:43.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:42 smithi100 ceph-mon[39557]: pgmap v757: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:44.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:43 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:44.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:43 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:44.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:43 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:45.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:44 smithi100 ceph-mon[30138]: pgmap v758: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:45.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:44 smithi100 ceph-mon[39557]: pgmap v758: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:44 smithi179 ceph-mon[33633]: pgmap v758: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:46.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:46.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:46.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-94"}]: dispatch 2024-02-14T17:56:46.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-94"}]: dispatch 2024-02-14T17:56:46.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:46.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:46.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-94"}]: dispatch 2024-02-14T17:56:46.152 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-94"}]: dispatch 2024-02-14T17:56:46.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:46.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:56:46.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-94"}]: dispatch 2024-02-14T17:56:46.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-94"}]: dispatch 2024-02-14T17:56:47.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:46 smithi100 ceph-mon[30138]: pgmap v759: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:47.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:46 smithi100 ceph-mon[30138]: osdmap e495: 8 total, 8 up, 8 in 2024-02-14T17:56:47.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:46 smithi100 ceph-mon[39557]: pgmap v759: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:47.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:46 smithi100 ceph-mon[39557]: osdmap e495: 8 total, 8 up, 8 in 2024-02-14T17:56:47.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:46 smithi179 ceph-mon[33633]: pgmap v759: 324 pgs: 324 active+clean; 8.4 MiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:56:47.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:46 smithi179 ceph-mon[33633]: osdmap e495: 8 total, 8 up, 8 in 2024-02-14T17:56:48.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:47 smithi100 ceph-mon[39557]: osdmap e496: 8 total, 8 up, 8 in 2024-02-14T17:56:48.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:48.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:48.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:47 smithi100 ceph-mon[30138]: osdmap e496: 8 total, 8 up, 8 in 2024-02-14T17:56:48.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:48.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:48.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:47 smithi179 ceph-mon[33633]: osdmap e496: 8 total, 8 up, 8 in 2024-02-14T17:56:48.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:48.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:56:49.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:48 smithi100 ceph-mon[39557]: pgmap v762: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:49.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:48 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:49.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:49.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:48 smithi100 ceph-mon[39557]: osdmap e497: 8 total, 8 up, 8 in 2024-02-14T17:56:49.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:49.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:48 smithi100 ceph-mon[30138]: pgmap v762: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:49.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:48 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:49.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:49.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:48 smithi100 ceph-mon[30138]: osdmap e497: 8 total, 8 up, 8 in 2024-02-14T17:56:49.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:48 smithi179 ceph-mon[33633]: pgmap v762: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:48 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:56:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:48 smithi179 ceph-mon[33633]: osdmap e497: 8 total, 8 up, 8 in 2024-02-14T17:56:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:56:50.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:49 smithi100 ceph-mon[30138]: osdmap e498: 8 total, 8 up, 8 in 2024-02-14T17:56:50.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:49 smithi100 ceph-mon[39557]: osdmap e498: 8 total, 8 up, 8 in 2024-02-14T17:56:50.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:49 smithi179 ceph-mon[33633]: osdmap e498: 8 total, 8 up, 8 in 2024-02-14T17:56:50.859 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:56:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:56:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:56:51.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:50 smithi100 ceph-mon[30138]: pgmap v765: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:51.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:50 smithi100 ceph-mon[39557]: pgmap v765: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:50 smithi179 ceph-mon[33633]: pgmap v765: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:56:52.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:56:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:56:53.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:52 smithi100 ceph-mon[30138]: pgmap v766: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-14T17:56:53.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:52 smithi100 ceph-mon[39557]: pgmap v766: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-14T17:56:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:52 smithi179 ceph-mon[33633]: pgmap v766: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-14T17:56:54.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:53 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:54.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:53 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:53 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:54.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:53 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:53 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:56:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:53 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:56:55.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:54 smithi100 ceph-mon[30138]: pgmap v767: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 741 B/s rd, 741 B/s wr, 2 op/s 2024-02-14T17:56:55.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:54 smithi100 ceph-mon[39557]: pgmap v767: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 741 B/s rd, 741 B/s wr, 2 op/s 2024-02-14T17:56:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:54 smithi179 ceph-mon[33633]: pgmap v767: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 741 B/s rd, 741 B/s wr, 2 op/s 2024-02-14T17:56:57.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:56 smithi100 ceph-mon[30138]: pgmap v768: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-02-14T17:56:57.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:56 smithi100 ceph-mon[39557]: pgmap v768: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-02-14T17:56:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:56 smithi179 ceph-mon[33633]: pgmap v768: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-02-14T17:56:58.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:57 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:58.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:57 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:57 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:56:59.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:58 smithi100 ceph-mon[30138]: pgmap v769: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 517 B/s wr, 2 op/s 2024-02-14T17:56:59.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:58 smithi100 ceph-mon[39557]: pgmap v769: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 517 B/s wr, 2 op/s 2024-02-14T17:56:59.150 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 17:56:58 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1[49905]: 2024-02-14T17:56:58.885+0000 7ff678a1f700 -1 snap_mapper.add_oid found existing snaps mapped on 106:4280a0a0:test-rados-api-smithi100-79838-97::foo:21, removing 2024-02-14T17:56:59.150 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 17:56:58 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3[62804]: 2024-02-14T17:56:58.885+0000 7f58ce9d1700 -1 snap_mapper.add_oid found existing snaps mapped on 106:4280a0a0:test-rados-api-smithi100-79838-97::foo:21, removing 2024-02-14T17:56:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:58 smithi179 ceph-mon[33633]: pgmap v769: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 517 B/s wr, 2 op/s 2024-02-14T17:56:59.321 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 17:56:58 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T17:56:58.884+0000 7f076a749700 -1 snap_mapper.add_oid found existing snaps mapped on 106:4280a0a0:test-rados-api-smithi100-79838-97::foo:21, removing 2024-02-14T17:57:00.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:00.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:00.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-96"}]: dispatch 2024-02-14T17:57:00.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:56:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-96"}]: dispatch 2024-02-14T17:57:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-96"}]: dispatch 2024-02-14T17:57:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:56:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-96"}]: dispatch 2024-02-14T17:57:00.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:00.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:00.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-96"}]: dispatch 2024-02-14T17:57:00.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:56:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-96"}]: dispatch 2024-02-14T17:57:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:57:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:57:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:00 smithi179 ceph-mon[33633]: pgmap v770: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 470 B/s wr, 1 op/s 2024-02-14T17:57:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:00 smithi179 ceph-mon[33633]: osdmap e499: 8 total, 8 up, 8 in 2024-02-14T17:57:01.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:00 smithi100 ceph-mon[30138]: pgmap v770: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 470 B/s wr, 1 op/s 2024-02-14T17:57:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:00 smithi100 ceph-mon[30138]: osdmap e499: 8 total, 8 up, 8 in 2024-02-14T17:57:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:00 smithi100 ceph-mon[39557]: pgmap v770: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 470 B/s wr, 1 op/s 2024-02-14T17:57:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:00 smithi100 ceph-mon[39557]: osdmap e499: 8 total, 8 up, 8 in 2024-02-14T17:57:02.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:57:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:57:02.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:01 smithi179 ceph-mon[33633]: osdmap e500: 8 total, 8 up, 8 in 2024-02-14T17:57:02.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:02.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:02.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:02.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:01 smithi179 ceph-mon[33633]: osdmap e501: 8 total, 8 up, 8 in 2024-02-14T17:57:02.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:01 smithi100 ceph-mon[30138]: osdmap e500: 8 total, 8 up, 8 in 2024-02-14T17:57:02.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:02.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:02.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:02.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:01 smithi100 ceph-mon[30138]: osdmap e501: 8 total, 8 up, 8 in 2024-02-14T17:57:02.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:01 smithi100 ceph-mon[39557]: osdmap e500: 8 total, 8 up, 8 in 2024-02-14T17:57:02.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:02.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:02.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:02.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:01 smithi100 ceph-mon[39557]: osdmap e501: 8 total, 8 up, 8 in 2024-02-14T17:57:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:02 smithi179 ceph-mon[33633]: pgmap v773: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-14T17:57:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:02 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:02 smithi179 ceph-mon[33633]: osdmap e502: 8 total, 8 up, 8 in 2024-02-14T17:57:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-98", "mode": "writeback"}]: dispatch 2024-02-14T17:57:03.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-98", "mode": "writeback"}]: dispatch 2024-02-14T17:57:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[39557]: pgmap v773: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[39557]: osdmap e502: 8 total, 8 up, 8 in 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-98", "mode": "writeback"}]: dispatch 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-98", "mode": "writeback"}]: dispatch 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[30138]: pgmap v773: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:03.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:03.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[30138]: osdmap e502: 8 total, 8 up, 8 in 2024-02-14T17:57:03.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-98", "mode": "writeback"}]: dispatch 2024-02-14T17:57:03.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-98", "mode": "writeback"}]: dispatch 2024-02-14T17:57:04.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:03 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:03 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-98", "mode": "writeback"}]': finished 2024-02-14T17:57:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:03 smithi179 ceph-mon[33633]: osdmap e503: 8 total, 8 up, 8 in 2024-02-14T17:57:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:04.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:04.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:04.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:04.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-98", "mode": "writeback"}]': finished 2024-02-14T17:57:04.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[39557]: osdmap e503: 8 total, 8 up, 8 in 2024-02-14T17:57:04.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:04.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-98", "mode": "writeback"}]': finished 2024-02-14T17:57:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[30138]: osdmap e503: 8 total, 8 up, 8 in 2024-02-14T17:57:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:04 smithi179 ceph-mon[33633]: pgmap v776: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-14T17:57:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-98"}]': finished 2024-02-14T17:57:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:04 smithi179 ceph-mon[33633]: osdmap e504: 8 total, 8 up, 8 in 2024-02-14T17:57:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:05.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:04 smithi100 ceph-mon[39557]: pgmap v776: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-14T17:57:05.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-98"}]': finished 2024-02-14T17:57:05.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:04 smithi100 ceph-mon[39557]: osdmap e504: 8 total, 8 up, 8 in 2024-02-14T17:57:05.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:04 smithi100 ceph-mon[30138]: pgmap v776: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-02-14T17:57:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-98"}]': finished 2024-02-14T17:57:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:04 smithi100 ceph-mon[30138]: osdmap e504: 8 total, 8 up, 8 in 2024-02-14T17:57:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T17:57:06.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:05 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:06.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:57:06.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:05 smithi179 ceph-mon[33633]: osdmap e505: 8 total, 8 up, 8 in 2024-02-14T17:57:06.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:57:06.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:57:06.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:05 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:06.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:57:06.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:05 smithi100 ceph-mon[39557]: osdmap e505: 8 total, 8 up, 8 in 2024-02-14T17:57:06.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:57:06.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:57:06.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:05 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:06.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T17:57:06.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:05 smithi100 ceph-mon[30138]: osdmap e505: 8 total, 8 up, 8 in 2024-02-14T17:57:06.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:57:06.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-14T17:57:06.529 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478bfc0 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478c6a0 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478c200 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478c330 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478c440 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478c590 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478cc70 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478c7d0 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478c900 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478ca10 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478cb60 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478d260 2024-02-14T17:57:06.530 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478cdc0 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478cf10 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478d040 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478d3b0 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478d840 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478d150 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478b650 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478b780 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478de10 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478d970 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478da80 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478dbd0 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478df60 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478e410 2024-02-14T17:57:06.531 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478e520 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478dd00 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478e090 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478e1e0 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478e650 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478e780 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478ed70 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478e8d0 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478ea20 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478eb50 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478eec0 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478f350 2024-02-14T17:57:06.532 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478f480 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478d4e0 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478d610 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478ec60 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478d720 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478fa50 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478f5b0 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478f700 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478f810 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478f920 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c4790040 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478fba0 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 0x5648c478fcd0 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.533 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.534 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.534 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.534 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.534 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.534 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.534 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.534 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.534 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.534 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.534 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.535 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:06.536 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:57:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:06 smithi179 ceph-mon[33633]: pgmap v779: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_count","val": "1"}]': finished 2024-02-14T17:57:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:06 smithi179 ceph-mon[33633]: osdmap e506: 8 total, 8 up, 8 in 2024-02-14T17:57:07.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:07.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:07.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:06 smithi100 ceph-mon[39557]: pgmap v779: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_count","val": "1"}]': finished 2024-02-14T17:57:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:06 smithi100 ceph-mon[39557]: osdmap e506: 8 total, 8 up, 8 in 2024-02-14T17:57:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:06 smithi100 ceph-mon[30138]: pgmap v779: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_count","val": "1"}]': finished 2024-02-14T17:57:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:06 smithi100 ceph-mon[30138]: osdmap e506: 8 total, 8 up, 8 in 2024-02-14T17:57:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:07.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:08 smithi179 ceph-mon[33633]: pgmap v782: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:08 smithi179 ceph-mon[33633]: osdmap e507: 8 total, 8 up, 8 in 2024-02-14T17:57:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:57:09.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:57:09.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:08 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[39557]: pgmap v782: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[39557]: osdmap e507: 8 total, 8 up, 8 in 2024-02-14T17:57:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:57:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:57:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[30138]: pgmap v782: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[30138]: osdmap e507: 8 total, 8 up, 8 in 2024-02-14T17:57:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:57:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-14T17:57:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:08 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:10.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "target_max_objects","val": "250"}]': finished 2024-02-14T17:57:10.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:09 smithi100 ceph-mon[39557]: osdmap e508: 8 total, 8 up, 8 in 2024-02-14T17:57:10.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:10.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:09 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:10.300 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "target_max_objects","val": "250"}]': finished 2024-02-14T17:57:10.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:09 smithi100 ceph-mon[30138]: osdmap e508: 8 total, 8 up, 8 in 2024-02-14T17:57:10.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:10.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:09 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-98","var": "target_max_objects","val": "250"}]': finished 2024-02-14T17:57:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:09 smithi179 ceph-mon[33633]: osdmap e508: 8 total, 8 up, 8 in 2024-02-14T17:57:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:09 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:10.649 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:57:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:10 smithi179 ceph-mon[33633]: pgmap v785: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:57:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:10 smithi179 ceph-mon[33633]: osdmap e509: 8 total, 8 up, 8 in 2024-02-14T17:57:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:11.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:10 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:11.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-mon[39557]: pgmap v785: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:11.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:57:11.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-mon[39557]: osdmap e509: 8 total, 8 up, 8 in 2024-02-14T17:57:11.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:11.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:11.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-mon[30138]: pgmap v785: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:57:11.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:57:11.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-mon[30138]: osdmap e509: 8 total, 8 up, 8 in 2024-02-14T17:57:11.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:11.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:10 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98"}]: dispatch 2024-02-14T17:57:12.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:57:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:57:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:11 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98"}]': finished 2024-02-14T17:57:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:11 smithi179 ceph-mon[33633]: osdmap e510: 8 total, 8 up, 8 in 2024-02-14T17:57:12.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:11 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:12.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:11 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98"}]': finished 2024-02-14T17:57:12.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:11 smithi100 ceph-mon[39557]: osdmap e510: 8 total, 8 up, 8 in 2024-02-14T17:57:12.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:11 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:12.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:11 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-98"}]': finished 2024-02-14T17:57:12.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:11 smithi100 ceph-mon[30138]: osdmap e510: 8 total, 8 up, 8 in 2024-02-14T17:57:12.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:11 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:12 smithi179 ceph-mon[33633]: pgmap v788: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:57:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:12 smithi179 ceph-mon[33633]: osdmap e511: 8 total, 8 up, 8 in 2024-02-14T17:57:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:12 smithi100 ceph-mon[39557]: pgmap v788: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:57:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:12 smithi100 ceph-mon[39557]: osdmap e511: 8 total, 8 up, 8 in 2024-02-14T17:57:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:12 smithi100 ceph-mon[30138]: pgmap v788: 324 pgs: 324 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:57:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:12 smithi100 ceph-mon[30138]: osdmap e511: 8 total, 8 up, 8 in 2024-02-14T17:57:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:13 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:13 smithi179 ceph-mon[33633]: osdmap e512: 8 total, 8 up, 8 in 2024-02-14T17:57:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:14.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:13 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:14.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:13 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:14.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:13 smithi100 ceph-mon[30138]: osdmap e512: 8 total, 8 up, 8 in 2024-02-14T17:57:14.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:14.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:13 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:13 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:13 smithi100 ceph-mon[39557]: osdmap e512: 8 total, 8 up, 8 in 2024-02-14T17:57:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:13 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:14 smithi179 ceph-mon[33633]: pgmap v791: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:14 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:14 smithi179 ceph-mon[33633]: osdmap e513: 8 total, 8 up, 8 in 2024-02-14T17:57:15.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:15.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:14 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[30138]: pgmap v791: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[30138]: osdmap e513: 8 total, 8 up, 8 in 2024-02-14T17:57:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[39557]: pgmap v791: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[39557]: osdmap e513: 8 total, 8 up, 8 in 2024-02-14T17:57:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:15.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:14 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:57:16.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:16.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:15 smithi179 ceph-mon[33633]: osdmap e514: 8 total, 8 up, 8 in 2024-02-14T17:57:16.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:16.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:15 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:16.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:16.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:15 smithi100 ceph-mon[30138]: osdmap e514: 8 total, 8 up, 8 in 2024-02-14T17:57:16.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:16.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:15 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:16.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:57:16.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:15 smithi100 ceph-mon[39557]: osdmap e514: 8 total, 8 up, 8 in 2024-02-14T17:57:16.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:16.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:15 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:16 smithi179 ceph-mon[33633]: pgmap v794: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-100"}]': finished 2024-02-14T17:57:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:16 smithi179 ceph-mon[33633]: osdmap e515: 8 total, 8 up, 8 in 2024-02-14T17:57:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-100", "mode": "writeback"}]: dispatch 2024-02-14T17:57:17.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:16 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-100", "mode": "writeback"}]: dispatch 2024-02-14T17:57:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:16 smithi100 ceph-mon[39557]: pgmap v794: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-100"}]': finished 2024-02-14T17:57:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:16 smithi100 ceph-mon[39557]: osdmap e515: 8 total, 8 up, 8 in 2024-02-14T17:57:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-100", "mode": "writeback"}]: dispatch 2024-02-14T17:57:17.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:16 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-100", "mode": "writeback"}]: dispatch 2024-02-14T17:57:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:16 smithi100 ceph-mon[30138]: pgmap v794: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 665 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-6", "overlaypool": "test-rados-api-smithi100-79838-100"}]': finished 2024-02-14T17:57:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:16 smithi100 ceph-mon[30138]: osdmap e515: 8 total, 8 up, 8 in 2024-02-14T17:57:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-100", "mode": "writeback"}]: dispatch 2024-02-14T17:57:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:16 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-100", "mode": "writeback"}]: dispatch 2024-02-14T17:57:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:17 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-100", "mode": "writeback"}]': finished 2024-02-14T17:57:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:17 smithi179 ceph-mon[33633]: osdmap e516: 8 total, 8 up, 8 in 2024-02-14T17:57:18.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:18.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:17 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:18.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:17 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-100", "mode": "writeback"}]': finished 2024-02-14T17:57:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:17 smithi100 ceph-mon[39557]: osdmap e516: 8 total, 8 up, 8 in 2024-02-14T17:57:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:18.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:17 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:17 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:18.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-100", "mode": "writeback"}]': finished 2024-02-14T17:57:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:17 smithi100 ceph-mon[30138]: osdmap e516: 8 total, 8 up, 8 in 2024-02-14T17:57:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:18.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:17 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:19 smithi179 ceph-mon[33633]: pgmap v797: 324 pgs: 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:19 smithi179 ceph-mon[33633]: osdmap e517: 8 total, 8 up, 8 in 2024-02-14T17:57:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:19.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:19 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:19.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:19 smithi100 ceph-mon[39557]: pgmap v797: 324 pgs: 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:19.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:19.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:19 smithi100 ceph-mon[39557]: osdmap e517: 8 total, 8 up, 8 in 2024-02-14T17:57:19.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:19.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:19 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:19.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:19 smithi100 ceph-mon[30138]: pgmap v797: 324 pgs: 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:19.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:19.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:19 smithi100 ceph-mon[30138]: osdmap e517: 8 total, 8 up, 8 in 2024-02-14T17:57:19.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:19.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:19 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:20 smithi179 ceph-mon[33633]: osdmap e518: 8 total, 8 up, 8 in 2024-02-14T17:57:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:20.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:20 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:20.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:20 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:57:20.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:20.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[30138]: osdmap e518: 8 total, 8 up, 8 in 2024-02-14T17:57:20.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:20.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:20.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:20.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:57:20.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:20.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[39557]: osdmap e518: 8 total, 8 up, 8 in 2024-02-14T17:57:20.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:20.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:20.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:20.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:57:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:57:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:57:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:21 smithi179 ceph-mon[33633]: pgmap v800: 324 pgs: 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:21 smithi179 ceph-mon[33633]: osdmap e519: 8 total, 8 up, 8 in 2024-02-14T17:57:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:57:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:57:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:21 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-14T17:57:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:21 smithi179 ceph-mon[33633]: osdmap e520: 8 total, 8 up, 8 in 2024-02-14T17:57:21.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[30138]: pgmap v800: 324 pgs: 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[30138]: osdmap e519: 8 total, 8 up, 8 in 2024-02-14T17:57:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:57:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:57:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-14T17:57:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[30138]: osdmap e520: 8 total, 8 up, 8 in 2024-02-14T17:57:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[39557]: pgmap v800: 324 pgs: 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[39557]: osdmap e519: 8 total, 8 up, 8 in 2024-02-14T17:57:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:57:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-14T17:57:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-14T17:57:21.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:21 smithi100 ceph-mon[39557]: osdmap e520: 8 total, 8 up, 8 in 2024-02-14T17:57:22.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:57:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:57:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:22 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:22.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:22.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:22 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:22 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:23 smithi179 ceph-mon[33633]: pgmap v803: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:57:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:23 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:57:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:23 smithi179 ceph-mon[33633]: osdmap e521: 8 total, 8 up, 8 in 2024-02-14T17:57:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:23 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:23 smithi100 ceph-mon[39557]: pgmap v803: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:57:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:23 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:57:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:23 smithi100 ceph-mon[39557]: osdmap e521: 8 total, 8 up, 8 in 2024-02-14T17:57:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:23 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:23.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:23 smithi100 ceph-mon[30138]: pgmap v803: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:57:23.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:23 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]': finished 2024-02-14T17:57:23.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:23 smithi100 ceph-mon[30138]: osdmap e521: 8 total, 8 up, 8 in 2024-02-14T17:57:23.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:23.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:23 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100"}]: dispatch 2024-02-14T17:57:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:24 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100"}]': finished 2024-02-14T17:57:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:24 smithi179 ceph-mon[33633]: osdmap e522: 8 total, 8 up, 8 in 2024-02-14T17:57:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:24 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:24 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:57:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:24 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100"}]': finished 2024-02-14T17:57:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:24 smithi100 ceph-mon[39557]: osdmap e522: 8 total, 8 up, 8 in 2024-02-14T17:57:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:24 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:24.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:24 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:57:24.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:24.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:24 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-100"}]': finished 2024-02-14T17:57:24.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:24 smithi100 ceph-mon[30138]: osdmap e522: 8 total, 8 up, 8 in 2024-02-14T17:57:24.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:24 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:24.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:24 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:57:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:25 smithi179 ceph-mon[33633]: pgmap v806: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:57:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:25 smithi179 ceph-mon[33633]: osdmap e523: 8 total, 8 up, 8 in 2024-02-14T17:57:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:25 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:57:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:25 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:57:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:25 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:57:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:25 smithi100 ceph-mon[30138]: pgmap v806: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:57:25.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:25 smithi100 ceph-mon[30138]: osdmap e523: 8 total, 8 up, 8 in 2024-02-14T17:57:25.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:25 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:57:25.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:25 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:57:25.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:25 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:57:25.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:25 smithi100 ceph-mon[39557]: pgmap v806: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T17:57:25.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:25 smithi100 ceph-mon[39557]: osdmap e523: 8 total, 8 up, 8 in 2024-02-14T17:57:25.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:25 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:57:25.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:25 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:57:25.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:25 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:57:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:26 smithi179 ceph-mon[33633]: osdmap e524: 8 total, 8 up, 8 in 2024-02-14T17:57:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:26.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:26.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:26 smithi100 ceph-mon[30138]: osdmap e524: 8 total, 8 up, 8 in 2024-02-14T17:57:26.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:26.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:26.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:26 smithi100 ceph-mon[39557]: osdmap e524: 8 total, 8 up, 8 in 2024-02-14T17:57:26.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:26.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:27 smithi179 ceph-mon[33633]: pgmap v809: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:27 smithi179 ceph-mon[33633]: osdmap e525: 8 total, 8 up, 8 in 2024-02-14T17:57:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:57:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:57:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:57:27.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:27 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[30138]: pgmap v809: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[30138]: osdmap e525: 8 total, 8 up, 8 in 2024-02-14T17:57:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:57:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:57:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:57:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[39557]: pgmap v809: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[39557]: osdmap e525: 8 total, 8 up, 8 in 2024-02-14T17:57:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-14T17:57:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:57:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-14T17:57:27.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:27 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:28.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:57:28.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:28 smithi100 ceph-mon[39557]: osdmap e526: 8 total, 8 up, 8 in 2024-02-14T17:57:28.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:57:28.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:57:28.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:57:28.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:28 smithi100 ceph-mon[30138]: osdmap e526: 8 total, 8 up, 8 in 2024-02-14T17:57:28.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:57:28.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:57:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-14T17:57:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:28 smithi179 ceph-mon[33633]: osdmap e526: 8 total, 8 up, 8 in 2024-02-14T17:57:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:57:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-14T17:57:29.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:29 smithi100 ceph-mon[39557]: pgmap v812: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:29.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:57:29.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:29 smithi100 ceph-mon[39557]: osdmap e527: 8 total, 8 up, 8 in 2024-02-14T17:57:29.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:57:29.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:29 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:57:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:29 smithi100 ceph-mon[30138]: pgmap v812: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:29.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:57:29.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:29 smithi100 ceph-mon[30138]: osdmap e527: 8 total, 8 up, 8 in 2024-02-14T17:57:29.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:57:29.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:29 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:57:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:29 smithi179 ceph-mon[33633]: pgmap v812: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-14T17:57:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:29 smithi179 ceph-mon[33633]: osdmap e527: 8 total, 8 up, 8 in 2024-02-14T17:57:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:57:29.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:29 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-14T17:57:30.087 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:57:30.087 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[39557]: osdmap e528: 8 total, 8 up, 8 in 2024-02-14T17:57:30.087 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:30.087 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:30.087 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-102"}]: dispatch 2024-02-14T17:57:30.087 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-102"}]: dispatch 2024-02-14T17:57:30.088 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:57:30.088 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[30138]: osdmap e528: 8 total, 8 up, 8 in 2024-02-14T17:57:30.088 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:30.088 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:30.088 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-102"}]: dispatch 2024-02-14T17:57:30.088 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-102"}]: dispatch 2024-02-14T17:57:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-14T17:57:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:30 smithi179 ceph-mon[33633]: osdmap e528: 8 total, 8 up, 8 in 2024-02-14T17:57:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:30.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-6"}]: dispatch 2024-02-14T17:57:30.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3090284545' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-102"}]: dispatch 2024-02-14T17:57:30.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:30 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-6", "tierpool": "test-rados-api-smithi100-79838-102"}]: dispatch 2024-02-14T17:57:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:57:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:57:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:31.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:31 smithi100 ceph-mon[30138]: pgmap v815: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:31.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:31 smithi100 ceph-mon[30138]: osdmap e529: 8 total, 8 up, 8 in 2024-02-14T17:57:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:31 smithi100 ceph-mon[39557]: pgmap v815: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:31 smithi100 ceph-mon[39557]: osdmap e529: 8 total, 8 up, 8 in 2024-02-14T17:57:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:31 smithi179 ceph-mon[33633]: pgmap v815: 324 pgs: 324 active+clean; 8.4 MiB data, 667 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:31 smithi179 ceph-mon[33633]: osdmap e529: 8 total, 8 up, 8 in 2024-02-14T17:57:32.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[39557]: osdmap e530: 8 total, 8 up, 8 in 2024-02-14T17:57:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[39557]: pgmap v818: 260 pgs: 260 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[30138]: osdmap e530: 8 total, 8 up, 8 in 2024-02-14T17:57:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:32.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:32.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:32 smithi100 ceph-mon[30138]: pgmap v818: 260 pgs: 260 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:32.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:57:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:57:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:32 smithi179 ceph-mon[33633]: osdmap e530: 8 total, 8 up, 8 in 2024-02-14T17:57:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:32 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:32 smithi179 ceph-mon[33633]: pgmap v818: 260 pgs: 260 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:57:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:33 smithi100 ceph-mon[39557]: osdmap e531: 8 total, 8 up, 8 in 2024-02-14T17:57:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-79838-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:33 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-79838-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:33.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:33 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:57:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:33 smithi100 ceph-mon[30138]: osdmap e531: 8 total, 8 up, 8 in 2024-02-14T17:57:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-79838-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:33 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-79838-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:33 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:33.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:57:33.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:33 smithi179 ceph-mon[33633]: osdmap e531: 8 total, 8 up, 8 in 2024-02-14T17:57:33.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-79838-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:33.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:33 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-79838-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:57:33.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:33 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:34 smithi100 ceph-mon[39557]: osdmap e532: 8 total, 8 up, 8 in 2024-02-14T17:57:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:34 smithi100 ceph-mon[39557]: pgmap v821: 260 pgs: 260 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:34 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-79838-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-79838-104"}]': finished 2024-02-14T17:57:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:34 smithi100 ceph-mon[39557]: osdmap e533: 8 total, 8 up, 8 in 2024-02-14T17:57:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:34 smithi100 ceph-mon[30138]: osdmap e532: 8 total, 8 up, 8 in 2024-02-14T17:57:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:34 smithi100 ceph-mon[30138]: pgmap v821: 260 pgs: 260 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:34 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-79838-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-79838-104"}]': finished 2024-02-14T17:57:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:34 smithi100 ceph-mon[30138]: osdmap e533: 8 total, 8 up, 8 in 2024-02-14T17:57:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:34 smithi179 ceph-mon[33633]: osdmap e532: 8 total, 8 up, 8 in 2024-02-14T17:57:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:34 smithi179 ceph-mon[33633]: pgmap v821: 260 pgs: 260 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:34 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi100-79838-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi100-79838-104"}]': finished 2024-02-14T17:57:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:34 smithi179 ceph-mon[33633]: osdmap e533: 8 total, 8 up, 8 in 2024-02-14T17:57:36.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:36 smithi100 ceph-mon[39557]: osdmap e534: 8 total, 8 up, 8 in 2024-02-14T17:57:36.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:36 smithi100 ceph-mon[39557]: pgmap v824: 268 pgs: 8 unknown, 260 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:36.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:36 smithi100 ceph-mon[30138]: osdmap e534: 8 total, 8 up, 8 in 2024-02-14T17:57:36.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:36 smithi100 ceph-mon[30138]: pgmap v824: 268 pgs: 8 unknown, 260 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:36 smithi179 ceph-mon[33633]: osdmap e534: 8 total, 8 up, 8 in 2024-02-14T17:57:36.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:36 smithi179 ceph-mon[33633]: pgmap v824: 268 pgs: 8 unknown, 260 active+clean; 582 KiB data, 668 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:37 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:37 smithi100 ceph-mon[39557]: osdmap e535: 8 total, 8 up, 8 in 2024-02-14T17:57:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:37 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:37 smithi100 ceph-mon[30138]: osdmap e535: 8 total, 8 up, 8 in 2024-02-14T17:57:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:37.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:37 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:37.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:37 smithi179 ceph-mon[33633]: osdmap e535: 8 total, 8 up, 8 in 2024-02-14T17:57:37.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:38.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:38.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:38 smithi100 ceph-mon[39557]: osdmap e536: 8 total, 8 up, 8 in 2024-02-14T17:57:38.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:38 smithi100 ceph-mon[39557]: pgmap v827: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:57:38.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:38 smithi100 ceph-mon[39557]: osdmap e537: 8 total, 8 up, 8 in 2024-02-14T17:57:38.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:38 smithi100 ceph-mon[30138]: osdmap e536: 8 total, 8 up, 8 in 2024-02-14T17:57:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:38 smithi100 ceph-mon[30138]: pgmap v827: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:57:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:38 smithi100 ceph-mon[30138]: osdmap e537: 8 total, 8 up, 8 in 2024-02-14T17:57:38.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:38 smithi179 ceph-mon[33633]: osdmap e536: 8 total, 8 up, 8 in 2024-02-14T17:57:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:38 smithi179 ceph-mon[33633]: pgmap v827: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:57:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:38 smithi179 ceph-mon[33633]: osdmap e537: 8 total, 8 up, 8 in 2024-02-14T17:57:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:40.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:40.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:40 smithi100 ceph-mon[39557]: osdmap e538: 8 total, 8 up, 8 in 2024-02-14T17:57:40.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]: dispatch 2024-02-14T17:57:40.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:40 smithi100 ceph-mon[39557]: pgmap v830: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:57:40.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:40.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:40 smithi100 ceph-mon[30138]: osdmap e538: 8 total, 8 up, 8 in 2024-02-14T17:57:40.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]: dispatch 2024-02-14T17:57:40.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:40 smithi100 ceph-mon[30138]: pgmap v830: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:57:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:40 smithi179 ceph-mon[33633]: osdmap e538: 8 total, 8 up, 8 in 2024-02-14T17:57:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]: dispatch 2024-02-14T17:57:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:40 smithi179 ceph-mon[33633]: pgmap v830: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 582 KiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T17:57:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:57:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:57:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:41 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]': finished 2024-02-14T17:57:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:41 smithi100 ceph-mon[30138]: osdmap e539: 8 total, 8 up, 8 in 2024-02-14T17:57:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:41 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-107", "overlaypool": "test-rados-api-smithi100-79838-107-cache"}]: dispatch 2024-02-14T17:57:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:41 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]': finished 2024-02-14T17:57:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:41 smithi100 ceph-mon[39557]: osdmap e539: 8 total, 8 up, 8 in 2024-02-14T17:57:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:41 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-107", "overlaypool": "test-rados-api-smithi100-79838-107-cache"}]: dispatch 2024-02-14T17:57:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:41 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]': finished 2024-02-14T17:57:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:41 smithi179 ceph-mon[33633]: osdmap e539: 8 total, 8 up, 8 in 2024-02-14T17:57:41.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:41 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-107", "overlaypool": "test-rados-api-smithi100-79838-107-cache"}]: dispatch 2024-02-14T17:57:42.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-107", "overlaypool": "test-rados-api-smithi100-79838-107-cache"}]': finished 2024-02-14T17:57:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[30138]: osdmap e540: 8 total, 8 up, 8 in 2024-02-14T17:57:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-107-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:57:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[30138]: pgmap v833: 332 pgs: 332 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-107-cache", "mode": "writeback"}]': finished 2024-02-14T17:57:42.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[30138]: osdmap e541: 8 total, 8 up, 8 in 2024-02-14T17:57:42.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-107", "overlaypool": "test-rados-api-smithi100-79838-107-cache"}]': finished 2024-02-14T17:57:42.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[39557]: osdmap e540: 8 total, 8 up, 8 in 2024-02-14T17:57:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-107-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:57:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[39557]: pgmap v833: 332 pgs: 332 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-107-cache", "mode": "writeback"}]': finished 2024-02-14T17:57:42.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:42 smithi100 ceph-mon[39557]: osdmap e541: 8 total, 8 up, 8 in 2024-02-14T17:57:42.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:57:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:57:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-107", "overlaypool": "test-rados-api-smithi100-79838-107-cache"}]': finished 2024-02-14T17:57:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:42 smithi179 ceph-mon[33633]: osdmap e540: 8 total, 8 up, 8 in 2024-02-14T17:57:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-107-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:57:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:42 smithi179 ceph-mon[33633]: pgmap v833: 332 pgs: 332 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:42 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-107-cache", "mode": "writeback"}]': finished 2024-02-14T17:57:42.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:42 smithi179 ceph-mon[33633]: osdmap e541: 8 total, 8 up, 8 in 2024-02-14T17:57:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-107"}]: dispatch 2024-02-14T17:57:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:43 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-107"}]: dispatch 2024-02-14T17:57:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:43 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-107"}]: dispatch 2024-02-14T17:57:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:43 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-107"}]': finished 2024-02-14T17:57:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:44 smithi179 ceph-mon[33633]: osdmap e542: 8 total, 8 up, 8 in 2024-02-14T17:57:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]: dispatch 2024-02-14T17:57:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:44 smithi179 ceph-mon[33633]: pgmap v836: 332 pgs: 332 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-107"}]': finished 2024-02-14T17:57:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:44 smithi100 ceph-mon[30138]: osdmap e542: 8 total, 8 up, 8 in 2024-02-14T17:57:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]: dispatch 2024-02-14T17:57:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:44 smithi100 ceph-mon[30138]: pgmap v836: 332 pgs: 332 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-107"}]': finished 2024-02-14T17:57:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:44 smithi100 ceph-mon[39557]: osdmap e542: 8 total, 8 up, 8 in 2024-02-14T17:57:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]: dispatch 2024-02-14T17:57:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:44 smithi100 ceph-mon[39557]: pgmap v836: 332 pgs: 332 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:45 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]': finished 2024-02-14T17:57:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:45 smithi179 ceph-mon[33633]: osdmap e543: 8 total, 8 up, 8 in 2024-02-14T17:57:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:45 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]': finished 2024-02-14T17:57:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:45 smithi100 ceph-mon[30138]: osdmap e543: 8 total, 8 up, 8 in 2024-02-14T17:57:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:45 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:45.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/713961543' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-107", "tierpool": "test-rados-api-smithi100-79838-107-cache"}]': finished 2024-02-14T17:57:45.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:45 smithi100 ceph-mon[39557]: osdmap e543: 8 total, 8 up, 8 in 2024-02-14T17:57:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:46 smithi179 ceph-mon[33633]: osdmap e544: 8 total, 8 up, 8 in 2024-02-14T17:57:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:46 smithi179 ceph-mon[33633]: pgmap v839: 300 pgs: 300 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:46.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:46 smithi100 ceph-mon[30138]: osdmap e544: 8 total, 8 up, 8 in 2024-02-14T17:57:46.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:46 smithi100 ceph-mon[30138]: pgmap v839: 300 pgs: 300 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:46.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:46 smithi100 ceph-mon[39557]: osdmap e544: 8 total, 8 up, 8 in 2024-02-14T17:57:46.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:46 smithi100 ceph-mon[39557]: pgmap v839: 300 pgs: 300 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:47 smithi179 ceph-mon[33633]: osdmap e545: 8 total, 8 up, 8 in 2024-02-14T17:57:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:47.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[30138]: osdmap e545: 8 total, 8 up, 8 in 2024-02-14T17:57:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[39557]: osdmap e545: 8 total, 8 up, 8 in 2024-02-14T17:57:47.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:47.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:47.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:57:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:57:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:48 smithi179 ceph-mon[33633]: osdmap e546: 8 total, 8 up, 8 in 2024-02-14T17:57:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:48 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:48 smithi179 ceph-mon[33633]: pgmap v842: 268 pgs: 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:48.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:57:48.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:48 smithi100 ceph-mon[30138]: osdmap e546: 8 total, 8 up, 8 in 2024-02-14T17:57:48.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:48 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:48 smithi100 ceph-mon[30138]: pgmap v842: 268 pgs: 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:48.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:57:48.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:48 smithi100 ceph-mon[39557]: osdmap e546: 8 total, 8 up, 8 in 2024-02-14T17:57:48.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:48.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:48 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:48.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:48 smithi100 ceph-mon[39557]: pgmap v842: 268 pgs: 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:49 smithi179 ceph-mon[33633]: osdmap e547: 8 total, 8 up, 8 in 2024-02-14T17:57:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:49 smithi100 ceph-mon[30138]: osdmap e547: 8 total, 8 up, 8 in 2024-02-14T17:57:49.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:49 smithi100 ceph-mon[39557]: osdmap e547: 8 total, 8 up, 8 in 2024-02-14T17:57:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:50 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:57:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:50 smithi100 ceph-mon[30138]: osdmap e548: 8 total, 8 up, 8 in 2024-02-14T17:57:50.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:50 smithi100 ceph-mon[30138]: pgmap v845: 276 pgs: 8 unknown, 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:50.494 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:50 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:57:50.495 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:50 smithi100 ceph-mon[39557]: osdmap e548: 8 total, 8 up, 8 in 2024-02-14T17:57:50.495 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:50 smithi100 ceph-mon[39557]: pgmap v845: 276 pgs: 8 unknown, 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:50.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:50 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:57:50.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:50 smithi179 ceph-mon[33633]: osdmap e548: 8 total, 8 up, 8 in 2024-02-14T17:57:50.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:50 smithi179 ceph-mon[33633]: pgmap v845: 276 pgs: 8 unknown, 268 active+clean; 582 KiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:57:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:57:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:57:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:51 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:51 smithi179 ceph-mon[33633]: osdmap e549: 8 total, 8 up, 8 in 2024-02-14T17:57:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:51 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:51 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:51 smithi100 ceph-mon[30138]: osdmap e549: 8 total, 8 up, 8 in 2024-02-14T17:57:51.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:51.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:51 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:51 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:57:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:51 smithi100 ceph-mon[39557]: osdmap e549: 8 total, 8 up, 8 in 2024-02-14T17:57:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:51 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:57:52.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:57:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:57:52.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:52 smithi179 ceph-mon[33633]: osdmap e550: 8 total, 8 up, 8 in 2024-02-14T17:57:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:52 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:52 smithi179 ceph-mon[33633]: pgmap v848: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:52.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:52.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:52 smithi100 ceph-mon[39557]: osdmap e550: 8 total, 8 up, 8 in 2024-02-14T17:57:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:52 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:52 smithi100 ceph-mon[39557]: pgmap v848: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:52.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:57:52.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:52 smithi100 ceph-mon[30138]: osdmap e550: 8 total, 8 up, 8 in 2024-02-14T17:57:52.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:52.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:52 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:52.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:52 smithi100 ceph-mon[30138]: pgmap v848: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:53 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]': finished 2024-02-14T17:57:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:53 smithi179 ceph-mon[33633]: osdmap e551: 8 total, 8 up, 8 in 2024-02-14T17:57:53.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-109", "overlaypool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:53.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:53 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-109", "overlaypool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:53 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]': finished 2024-02-14T17:57:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:53 smithi100 ceph-mon[39557]: osdmap e551: 8 total, 8 up, 8 in 2024-02-14T17:57:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-109", "overlaypool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:53 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-109", "overlaypool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:53 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:57:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]': finished 2024-02-14T17:57:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:53 smithi100 ceph-mon[30138]: osdmap e551: 8 total, 8 up, 8 in 2024-02-14T17:57:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-109", "overlaypool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:53.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:53 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-109", "overlaypool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:57:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-109", "overlaypool": "test-rados-api-smithi100-79838-109-cache"}]': finished 2024-02-14T17:57:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:54 smithi179 ceph-mon[33633]: osdmap e552: 8 total, 8 up, 8 in 2024-02-14T17:57:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:57:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:57:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:54 smithi179 ceph-mon[33633]: pgmap v851: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:54 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-109-cache", "mode": "writeback"}]': finished 2024-02-14T17:57:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:54 smithi179 ceph-mon[33633]: osdmap e553: 8 total, 8 up, 8 in 2024-02-14T17:57:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:54 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-109", "overlaypool": "test-rados-api-smithi100-79838-109-cache"}]': finished 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[39557]: osdmap e552: 8 total, 8 up, 8 in 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[39557]: pgmap v851: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-109-cache", "mode": "writeback"}]': finished 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[39557]: osdmap e553: 8 total, 8 up, 8 in 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-109", "overlaypool": "test-rados-api-smithi100-79838-109-cache"}]': finished 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[30138]: osdmap e552: 8 total, 8 up, 8 in 2024-02-14T17:57:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:57:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-109-cache", "mode": "writeback"}]: dispatch 2024-02-14T17:57:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[30138]: pgmap v851: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:57:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-109-cache", "mode": "writeback"}]': finished 2024-02-14T17:57:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[30138]: osdmap e553: 8 total, 8 up, 8 in 2024-02-14T17:57:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:54.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:54 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T17:57:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:56 smithi179 ceph-mon[33633]: osdmap e554: 8 total, 8 up, 8 in 2024-02-14T17:57:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:56 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:56 smithi179 ceph-mon[33633]: pgmap v854: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:56.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:56 smithi100 ceph-mon[39557]: osdmap e554: 8 total, 8 up, 8 in 2024-02-14T17:57:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:56 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:56 smithi100 ceph-mon[39557]: pgmap v854: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-14T17:57:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:56 smithi100 ceph-mon[30138]: osdmap e554: 8 total, 8 up, 8 in 2024-02-14T17:57:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:56.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:56 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T17:57:56.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:56 smithi100 ceph-mon[30138]: pgmap v854: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:57:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:57 smithi179 ceph-mon[33633]: osdmap e555: 8 total, 8 up, 8 in 2024-02-14T17:57:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:57.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:57 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:57.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:57 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:57:57.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:57 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:57.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[30138]: osdmap e555: 8 total, 8 up, 8 in 2024-02-14T17:57:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:57:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-14T17:57:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[39557]: osdmap e555: 8 total, 8 up, 8 in 2024-02-14T17:57:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T17:57:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:57:57.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:57 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:57:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:58 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:58.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:57:58.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:58 smithi179 ceph-mon[33633]: osdmap e556: 8 total, 8 up, 8 in 2024-02-14T17:57:58.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:57:58.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:58 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:57:58.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:58 smithi179 ceph-mon[33633]: pgmap v857: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[39557]: osdmap e556: 8 total, 8 up, 8 in 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[39557]: pgmap v857: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[30138]: osdmap e556: 8 total, 8 up, 8 in 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-14T17:57:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:58 smithi100 ceph-mon[30138]: pgmap v857: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:57:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-14T17:57:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:59 smithi179 ceph-mon[33633]: osdmap e557: 8 total, 8 up, 8 in 2024-02-14T17:57:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:57:59 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:59.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-14T17:57:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:59 smithi100 ceph-mon[39557]: osdmap e557: 8 total, 8 up, 8 in 2024-02-14T17:57:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:57:59 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-14T17:57:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:59 smithi100 ceph-mon[30138]: osdmap e557: 8 total, 8 up, 8 in 2024-02-14T17:57:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:57:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:57:59 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:58:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:00 smithi179 ceph-mon[33633]: osdmap e558: 8 total, 8 up, 8 in 2024-02-14T17:58:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:58:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:58:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:00 smithi179 ceph-mon[33633]: pgmap v860: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:00.649 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:58:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:58:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-mon[39557]: osdmap e558: 8 total, 8 up, 8 in 2024-02-14T17:58:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:58:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:58:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-mon[39557]: pgmap v860: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:58:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-mon[30138]: osdmap e558: 8 total, 8 up, 8 in 2024-02-14T17:58:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:58:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]: dispatch 2024-02-14T17:58:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:00 smithi100 ceph-mon[30138]: pgmap v860: 308 pgs: 308 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:01.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]': finished 2024-02-14T17:58:01.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:01 smithi179 ceph-mon[33633]: osdmap e559: 8 total, 8 up, 8 in 2024-02-14T17:58:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]': finished 2024-02-14T17:58:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:01 smithi100 ceph-mon[30138]: osdmap e559: 8 total, 8 up, 8 in 2024-02-14T17:58:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-109", "tierpool": "test-rados-api-smithi100-79838-109-cache"}]': finished 2024-02-14T17:58:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:01 smithi100 ceph-mon[39557]: osdmap e559: 8 total, 8 up, 8 in 2024-02-14T17:58:02.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:02 smithi179 ceph-mon[33633]: osdmap e560: 8 total, 8 up, 8 in 2024-02-14T17:58:02.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:02 smithi179 ceph-mon[33633]: pgmap v863: 276 pgs: 276 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:02.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:58:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:58:02.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:02 smithi100 ceph-mon[30138]: osdmap e560: 8 total, 8 up, 8 in 2024-02-14T17:58:02.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:02 smithi100 ceph-mon[30138]: pgmap v863: 276 pgs: 276 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:02 smithi100 ceph-mon[39557]: osdmap e560: 8 total, 8 up, 8 in 2024-02-14T17:58:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:02 smithi100 ceph-mon[39557]: pgmap v863: 276 pgs: 276 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:03 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:03 smithi179 ceph-mon[33633]: osdmap e561: 8 total, 8 up, 8 in 2024-02-14T17:58:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:03 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:03 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:03 smithi100 ceph-mon[30138]: osdmap e561: 8 total, 8 up, 8 in 2024-02-14T17:58:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:03 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:03 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:03 smithi100 ceph-mon[39557]: osdmap e561: 8 total, 8 up, 8 in 2024-02-14T17:58:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:03 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:58:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:04 smithi179 ceph-mon[33633]: osdmap e562: 8 total, 8 up, 8 in 2024-02-14T17:58:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:04.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:04 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:04.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:04 smithi179 ceph-mon[33633]: pgmap v866: 268 pgs: 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:58:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:04 smithi100 ceph-mon[39557]: osdmap e562: 8 total, 8 up, 8 in 2024-02-14T17:58:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:04 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:04 smithi100 ceph-mon[39557]: pgmap v866: 268 pgs: 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:58:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:04 smithi100 ceph-mon[30138]: osdmap e562: 8 total, 8 up, 8 in 2024-02-14T17:58:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1361033649' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:04 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]: dispatch 2024-02-14T17:58:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:04 smithi100 ceph-mon[30138]: pgmap v866: 268 pgs: 268 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:05 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:05 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:58:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:05 smithi179 ceph-mon[33633]: osdmap e563: 8 total, 8 up, 8 in 2024-02-14T17:58:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:05 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:05 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:58:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:05 smithi100 ceph-mon[39557]: osdmap e563: 8 total, 8 up, 8 in 2024-02-14T17:58:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:05 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:05 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-109"}]': finished 2024-02-14T17:58:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:05 smithi100 ceph-mon[30138]: osdmap e563: 8 total, 8 up, 8 in 2024-02-14T17:58:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:06 smithi179 ceph-mon[33633]: osdmap e564: 8 total, 8 up, 8 in 2024-02-14T17:58:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:06 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:06 smithi179 ceph-mon[33633]: pgmap v869: 260 pgs: 260 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:06.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:06 smithi100 ceph-mon[39557]: osdmap e564: 8 total, 8 up, 8 in 2024-02-14T17:58:06.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:06.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:06 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:06 smithi100 ceph-mon[39557]: pgmap v869: 260 pgs: 260 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:06 smithi100 ceph-mon[30138]: osdmap e564: 8 total, 8 up, 8 in 2024-02-14T17:58:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:06 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:06 smithi100 ceph-mon[30138]: pgmap v869: 260 pgs: 260 active+clean; 582 KiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]': finished 2024-02-14T17:58:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:07 smithi100 ceph-mon[30138]: osdmap e565: 8 total, 8 up, 8 in 2024-02-14T17:58:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:07 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]': finished 2024-02-14T17:58:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:07 smithi100 ceph-mon[39557]: osdmap e565: 8 total, 8 up, 8 in 2024-02-14T17:58:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:07 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi100-79838-104"}]': finished 2024-02-14T17:58:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:07 smithi179 ceph-mon[33633]: osdmap e565: 8 total, 8 up, 8 in 2024-02-14T17:58:07.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/453301824' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:07.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:07 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]: dispatch 2024-02-14T17:58:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]': finished 2024-02-14T17:58:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[39557]: osdmap e566: 8 total, 8 up, 8 in 2024-02-14T17:58:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:58:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[39557]: pgmap v872: 260 pgs: 260 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-14T17:58:08.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]': finished 2024-02-14T17:58:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[30138]: osdmap e566: 8 total, 8 up, 8 in 2024-02-14T17:58:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:58:08.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:08 smithi100 ceph-mon[30138]: pgmap v872: 260 pgs: 260 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-14T17:58:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:08 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi100-79838-104"}]': finished 2024-02-14T17:58:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:08 smithi179 ceph-mon[33633]: osdmap e566: 8 total, 8 up, 8 in 2024-02-14T17:58:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-14T17:58:08.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:08 smithi179 ceph-mon[33633]: pgmap v872: 260 pgs: 260 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-14T17:58:09.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:58:09.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:09 smithi100 ceph-mon[30138]: osdmap e567: 8 total, 8 up, 8 in 2024-02-14T17:58:09.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:09.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:58:09.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:09 smithi100 ceph-mon[39557]: osdmap e567: 8 total, 8 up, 8 in 2024-02-14T17:58:09.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:10.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi100-79838-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-14T17:58:10.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:09 smithi179 ceph-mon[33633]: osdmap e567: 8 total, 8 up, 8 in 2024-02-14T17:58:10.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:10.801 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:58:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:58:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:10.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:10 smithi100 ceph-mon[39557]: osdmap e568: 8 total, 8 up, 8 in 2024-02-14T17:58:10.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:10 smithi100 ceph-mon[39557]: pgmap v875: 260 pgs: 260 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:10.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:10.801 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:10 smithi100 ceph-mon[39557]: osdmap e569: 8 total, 8 up, 8 in 2024-02-14T17:58:10.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:10 smithi100 ceph-mon[30138]: osdmap e568: 8 total, 8 up, 8 in 2024-02-14T17:58:10.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:10 smithi100 ceph-mon[30138]: pgmap v875: 260 pgs: 260 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:10.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:10.802 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:10 smithi100 ceph-mon[30138]: osdmap e569: 8 total, 8 up, 8 in 2024-02-14T17:58:11.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:10 smithi179 ceph-mon[33633]: osdmap e568: 8 total, 8 up, 8 in 2024-02-14T17:58:11.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:10 smithi179 ceph-mon[33633]: pgmap v875: 260 pgs: 260 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:11.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi100-79838-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:11.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:10 smithi179 ceph-mon[33633]: osdmap e569: 8 total, 8 up, 8 in 2024-02-14T17:58:12.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:58:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:58:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:12 smithi179 ceph-mon[33633]: osdmap e570: 8 total, 8 up, 8 in 2024-02-14T17:58:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:12 smithi179 ceph-mon[33633]: pgmap v878: 300 pgs: 12 creating+peering, 27 unknown, 261 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:12 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:12 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:12.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:12 smithi100 ceph-mon[39557]: osdmap e570: 8 total, 8 up, 8 in 2024-02-14T17:58:12.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:12 smithi100 ceph-mon[39557]: pgmap v878: 300 pgs: 12 creating+peering, 27 unknown, 261 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:12 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:12 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:12 smithi100 ceph-mon[30138]: osdmap e570: 8 total, 8 up, 8 in 2024-02-14T17:58:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:12 smithi100 ceph-mon[30138]: pgmap v878: 300 pgs: 12 creating+peering, 27 unknown, 261 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:12 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:12 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:13 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:13 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:13 smithi100 ceph-mon[30138]: osdmap e571: 8 total, 8 up, 8 in 2024-02-14T17:58:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:13 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:13.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:13 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:13.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:13.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:13 smithi100 ceph-mon[39557]: osdmap e571: 8 total, 8 up, 8 in 2024-02-14T17:58:13.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:13 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:13 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:13 smithi179 ceph-mon[33633]: osdmap e571: 8 total, 8 up, 8 in 2024-02-14T17:58:13.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[30138]: osdmap e572: 8 total, 8 up, 8 in 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-112"}]: dispatch 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[30138]: pgmap v881: 300 pgs: 12 creating+peering, 27 unknown, 261 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-112"}]': finished 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[30138]: osdmap e573: 8 total, 8 up, 8 in 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[39557]: osdmap e572: 8 total, 8 up, 8 in 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-112"}]: dispatch 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[39557]: pgmap v881: 300 pgs: 12 creating+peering, 27 unknown, 261 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-112"}]': finished 2024-02-14T17:58:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:14 smithi100 ceph-mon[39557]: osdmap e573: 8 total, 8 up, 8 in 2024-02-14T17:58:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:14 smithi179 ceph-mon[33633]: osdmap e572: 8 total, 8 up, 8 in 2024-02-14T17:58:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-112"}]: dispatch 2024-02-14T17:58:14.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:14 smithi179 ceph-mon[33633]: pgmap v881: 300 pgs: 12 creating+peering, 27 unknown, 261 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:14.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-112"}]': finished 2024-02-14T17:58:14.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:14 smithi179 ceph-mon[33633]: osdmap e573: 8 total, 8 up, 8 in 2024-02-14T17:58:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:15.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:16.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:16.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:16 smithi100 ceph-mon[39557]: osdmap e574: 8 total, 8 up, 8 in 2024-02-14T17:58:16.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112"}]: dispatch 2024-02-14T17:58:16.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:16 smithi100 ceph-mon[39557]: pgmap v884: 300 pgs: 12 creating+peering, 27 unknown, 261 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:16.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:16.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:16 smithi100 ceph-mon[30138]: osdmap e574: 8 total, 8 up, 8 in 2024-02-14T17:58:16.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112"}]: dispatch 2024-02-14T17:58:16.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:16 smithi100 ceph-mon[30138]: pgmap v884: 300 pgs: 12 creating+peering, 27 unknown, 261 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:16 smithi179 ceph-mon[33633]: osdmap e574: 8 total, 8 up, 8 in 2024-02-14T17:58:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112"}]: dispatch 2024-02-14T17:58:16.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:16 smithi179 ceph-mon[33633]: pgmap v884: 300 pgs: 12 creating+peering, 27 unknown, 261 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:17.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112"}]': finished 2024-02-14T17:58:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:17 smithi100 ceph-mon[39557]: osdmap e575: 8 total, 8 up, 8 in 2024-02-14T17:58:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112"}]': finished 2024-02-14T17:58:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:17 smithi100 ceph-mon[30138]: osdmap e575: 8 total, 8 up, 8 in 2024-02-14T17:58:18.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-112"}]': finished 2024-02-14T17:58:18.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:17 smithi179 ceph-mon[33633]: osdmap e575: 8 total, 8 up, 8 in 2024-02-14T17:58:18.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:18 smithi100 ceph-mon[39557]: osdmap e576: 8 total, 8 up, 8 in 2024-02-14T17:58:18.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:18 smithi100 ceph-mon[39557]: pgmap v887: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:18 smithi100 ceph-mon[39557]: osdmap e577: 8 total, 8 up, 8 in 2024-02-14T17:58:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:18 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:18 smithi100 ceph-mon[30138]: osdmap e576: 8 total, 8 up, 8 in 2024-02-14T17:58:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:18 smithi100 ceph-mon[30138]: pgmap v887: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:18 smithi100 ceph-mon[30138]: osdmap e577: 8 total, 8 up, 8 in 2024-02-14T17:58:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:18 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:19.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:18 smithi179 ceph-mon[33633]: osdmap e576: 8 total, 8 up, 8 in 2024-02-14T17:58:19.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:18 smithi179 ceph-mon[33633]: pgmap v887: 268 pgs: 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:19.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:18 smithi179 ceph-mon[33633]: osdmap e577: 8 total, 8 up, 8 in 2024-02-14T17:58:19.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:19.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:18 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:20.649 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:58:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:58:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:20.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:20.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:20 smithi100 ceph-mon[39557]: osdmap e578: 8 total, 8 up, 8 in 2024-02-14T17:58:20.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:20.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:20 smithi100 ceph-mon[39557]: pgmap v890: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:20.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:20.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:20 smithi100 ceph-mon[30138]: osdmap e578: 8 total, 8 up, 8 in 2024-02-14T17:58:20.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:20.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:20 smithi100 ceph-mon[30138]: pgmap v890: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:20.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:20.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:20 smithi179 ceph-mon[33633]: osdmap e578: 8 total, 8 up, 8 in 2024-02-14T17:58:20.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:20.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:20 smithi179 ceph-mon[33633]: pgmap v890: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:21 smithi179 ceph-mon[33633]: osdmap e579: 8 total, 8 up, 8 in 2024-02-14T17:58:21.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-114"}]: dispatch 2024-02-14T17:58:21.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:21.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:21 smithi100 ceph-mon[39557]: osdmap e579: 8 total, 8 up, 8 in 2024-02-14T17:58:21.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-114"}]: dispatch 2024-02-14T17:58:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:21 smithi100 ceph-mon[30138]: osdmap e579: 8 total, 8 up, 8 in 2024-02-14T17:58:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-114"}]: dispatch 2024-02-14T17:58:22.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:58:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:58:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-114"}]': finished 2024-02-14T17:58:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:22 smithi179 ceph-mon[33633]: osdmap e580: 8 total, 8 up, 8 in 2024-02-14T17:58:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-114", "mode": "writeback"}]: dispatch 2024-02-14T17:58:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:22 smithi179 ceph-mon[33633]: pgmap v893: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:22 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-114", "mode": "writeback"}]': finished 2024-02-14T17:58:22.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:22 smithi179 ceph-mon[33633]: osdmap e581: 8 total, 8 up, 8 in 2024-02-14T17:58:22.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-114"}]': finished 2024-02-14T17:58:22.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[39557]: osdmap e580: 8 total, 8 up, 8 in 2024-02-14T17:58:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-114", "mode": "writeback"}]: dispatch 2024-02-14T17:58:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[39557]: pgmap v893: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-114", "mode": "writeback"}]': finished 2024-02-14T17:58:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[39557]: osdmap e581: 8 total, 8 up, 8 in 2024-02-14T17:58:22.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-114"}]': finished 2024-02-14T17:58:22.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[30138]: osdmap e580: 8 total, 8 up, 8 in 2024-02-14T17:58:22.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-114", "mode": "writeback"}]: dispatch 2024-02-14T17:58:22.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[30138]: pgmap v893: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:22.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-114", "mode": "writeback"}]': finished 2024-02-14T17:58:22.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:22 smithi100 ceph-mon[30138]: osdmap e581: 8 total, 8 up, 8 in 2024-02-14T17:58:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:23 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:23.847 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:23.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:23 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:23.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:23.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:23 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:23.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:24 smithi179 ceph-mon[33633]: osdmap e582: 8 total, 8 up, 8 in 2024-02-14T17:58:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114"}]: dispatch 2024-02-14T17:58:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:24 smithi179 ceph-mon[33633]: pgmap v896: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:24.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:24.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:24 smithi100 ceph-mon[39557]: osdmap e582: 8 total, 8 up, 8 in 2024-02-14T17:58:24.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114"}]: dispatch 2024-02-14T17:58:24.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:24 smithi100 ceph-mon[39557]: pgmap v896: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:24 smithi100 ceph-mon[30138]: osdmap e582: 8 total, 8 up, 8 in 2024-02-14T17:58:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114"}]: dispatch 2024-02-14T17:58:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:24 smithi100 ceph-mon[30138]: pgmap v896: 300 pgs: 300 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:58:26.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:25 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:26.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114"}]': finished 2024-02-14T17:58:26.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:25 smithi179 ceph-mon[33633]: osdmap e583: 8 total, 8 up, 8 in 2024-02-14T17:58:26.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:25 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:58:26.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:25 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:26.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114"}]': finished 2024-02-14T17:58:26.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:25 smithi100 ceph-mon[39557]: osdmap e583: 8 total, 8 up, 8 in 2024-02-14T17:58:26.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:25 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:58:26.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:25 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:26.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-114"}]': finished 2024-02-14T17:58:26.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:25 smithi100 ceph-mon[30138]: osdmap e583: 8 total, 8 up, 8 in 2024-02-14T17:58:26.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:25 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:58:26.776 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:26 smithi179 ceph-mon[33633]: osdmap e584: 8 total, 8 up, 8 in 2024-02-14T17:58:26.776 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:26 smithi179 ceph-mon[33633]: pgmap v899: 268 pgs: 268 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:26.776 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:26 smithi179 ceph-mon[33633]: osdmap e585: 8 total, 8 up, 8 in 2024-02-14T17:58:26.776 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:27.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:26 smithi100 ceph-mon[30138]: osdmap e584: 8 total, 8 up, 8 in 2024-02-14T17:58:27.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:26 smithi100 ceph-mon[30138]: pgmap v899: 268 pgs: 268 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:27.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:26 smithi100 ceph-mon[30138]: osdmap e585: 8 total, 8 up, 8 in 2024-02-14T17:58:27.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:27.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:26 smithi100 ceph-mon[39557]: osdmap e584: 8 total, 8 up, 8 in 2024-02-14T17:58:27.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:26 smithi100 ceph-mon[39557]: pgmap v899: 268 pgs: 268 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:27.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:26 smithi100 ceph-mon[39557]: osdmap e585: 8 total, 8 up, 8 in 2024-02-14T17:58:27.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:27 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:27 smithi179 ceph-mon[33633]: osdmap e586: 8 total, 8 up, 8 in 2024-02-14T17:58:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:27 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:27 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:28.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:27 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:28.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:28.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:27 smithi100 ceph-mon[39557]: osdmap e586: 8 total, 8 up, 8 in 2024-02-14T17:58:28.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:27 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:28.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:27 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:27 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:28.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:28.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:27 smithi100 ceph-mon[30138]: osdmap e586: 8 total, 8 up, 8 in 2024-02-14T17:58:28.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:27 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:28.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:27 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:28.983 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:28 smithi179 ceph-mon[33633]: pgmap v902: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:28.983 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:28 smithi179 ceph-mon[33633]: osdmap e587: 8 total, 8 up, 8 in 2024-02-14T17:58:28.983 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:29.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:28 smithi100 ceph-mon[30138]: pgmap v902: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:28 smithi100 ceph-mon[30138]: osdmap e587: 8 total, 8 up, 8 in 2024-02-14T17:58:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:28 smithi100 ceph-mon[39557]: pgmap v902: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:28 smithi100 ceph-mon[39557]: osdmap e587: 8 total, 8 up, 8 in 2024-02-14T17:58:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:30 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:30 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:30 smithi179 ceph-mon[33633]: osdmap e588: 8 total, 8 up, 8 in 2024-02-14T17:58:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-116"}]: dispatch 2024-02-14T17:58:30.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:30 smithi179 ceph-mon[33633]: pgmap v905: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:30.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:30 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:58:30.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:30 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:58:30.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:30 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:30.649 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:58:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[39557]: osdmap e588: 8 total, 8 up, 8 in 2024-02-14T17:58:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-116"}]: dispatch 2024-02-14T17:58:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[39557]: pgmap v905: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:58:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:58:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[30138]: osdmap e588: 8 total, 8 up, 8 in 2024-02-14T17:58:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-116"}]: dispatch 2024-02-14T17:58:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[30138]: pgmap v905: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:58:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:58:30.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:30 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:31.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-116"}]': finished 2024-02-14T17:58:31.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:31 smithi179 ceph-mon[33633]: osdmap e589: 8 total, 8 up, 8 in 2024-02-14T17:58:31.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-116", "mode": "writeback"}]: dispatch 2024-02-14T17:58:31.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-116"}]': finished 2024-02-14T17:58:31.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:31 smithi100 ceph-mon[30138]: osdmap e589: 8 total, 8 up, 8 in 2024-02-14T17:58:31.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-116", "mode": "writeback"}]: dispatch 2024-02-14T17:58:31.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-116"}]': finished 2024-02-14T17:58:31.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:31 smithi100 ceph-mon[39557]: osdmap e589: 8 total, 8 up, 8 in 2024-02-14T17:58:31.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-116", "mode": "writeback"}]: dispatch 2024-02-14T17:58:32.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:58:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:58:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:32 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-116", "mode": "writeback"}]': finished 2024-02-14T17:58:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:32 smithi179 ceph-mon[33633]: osdmap e590: 8 total, 8 up, 8 in 2024-02-14T17:58:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "324.6"}]: dispatch 2024-02-14T17:58:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:32 smithi179 ceph-mon[33633]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "324.6"}]: dispatch 2024-02-14T17:58:32.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:32 smithi179 ceph-mon[33633]: pgmap v908: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:58:32.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:32.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-116", "mode": "writeback"}]': finished 2024-02-14T17:58:32.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[39557]: osdmap e590: 8 total, 8 up, 8 in 2024-02-14T17:58:32.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "324.6"}]: dispatch 2024-02-14T17:58:32.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[39557]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "324.6"}]: dispatch 2024-02-14T17:58:32.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[39557]: pgmap v908: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:58:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-116", "mode": "writeback"}]': finished 2024-02-14T17:58:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[30138]: osdmap e590: 8 total, 8 up, 8 in 2024-02-14T17:58:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "324.6"}]: dispatch 2024-02-14T17:58:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[30138]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "324.6"}]: dispatch 2024-02-14T17:58:32.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:32 smithi100 ceph-mon[30138]: pgmap v908: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:58:33.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:33 smithi179 ceph-mon[33633]: 324.6 scrub starts 2024-02-14T17:58:33.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:33 smithi179 ceph-mon[33633]: 324.6 scrub ok 2024-02-14T17:58:33.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:33 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:33.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:33 smithi100 ceph-mon[30138]: 324.6 scrub starts 2024-02-14T17:58:33.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:33 smithi100 ceph-mon[30138]: 324.6 scrub ok 2024-02-14T17:58:33.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:33 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:33.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:33 smithi100 ceph-mon[39557]: 324.6 scrub starts 2024-02-14T17:58:33.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:33 smithi100 ceph-mon[39557]: 324.6 scrub ok 2024-02-14T17:58:33.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:33 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:34 smithi179 ceph-mon[33633]: pgmap v909: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 960 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-14T17:58:34.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:34 smithi100 ceph-mon[30138]: pgmap v909: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 960 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-14T17:58:34.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:34 smithi100 ceph-mon[39557]: pgmap v909: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 960 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-02-14T17:58:37.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:36 smithi100 ceph-mon[30138]: pgmap v910: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 808 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-14T17:58:37.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:36 smithi100 ceph-mon[39557]: pgmap v910: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 808 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-14T17:58:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:36 smithi179 ceph-mon[33633]: pgmap v910: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 808 B/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-14T17:58:39.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:38 smithi100 ceph-mon[30138]: pgmap v911: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-14T17:58:39.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:38 smithi100 ceph-mon[39557]: pgmap v911: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-14T17:58:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:38 smithi179 ceph-mon[33633]: pgmap v911: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-14T17:58:40.835 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:58:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:58:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:41.104 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:40 smithi100 ceph-mon[39557]: pgmap v912: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 768 B/s wr, 2 op/s 2024-02-14T17:58:41.104 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:40 smithi100 ceph-mon[30138]: pgmap v912: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 768 B/s wr, 2 op/s 2024-02-14T17:58:41.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:40 smithi179 ceph-mon[33633]: pgmap v912: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 768 B/s wr, 2 op/s 2024-02-14T17:58:42.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:58:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:58:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:42 smithi179 ceph-mon[33633]: pgmap v913: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:42 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:42 smithi100 ceph-mon[39557]: pgmap v913: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:42 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:42 smithi100 ceph-mon[30138]: pgmap v913: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:58:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:42 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:58:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:44.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:43 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:44.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:43 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:44.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:43 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:44 smithi179 ceph-mon[33633]: pgmap v914: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-14T17:58:45.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:44 smithi100 ceph-mon[30138]: pgmap v914: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-14T17:58:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:44 smithi100 ceph-mon[39557]: pgmap v914: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-14T17:58:45.541 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.542 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.543 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.544 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.545 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.546 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.547 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.548 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.549 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:45.550 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T17:58:47.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:46 smithi179 ceph-mon[33633]: pgmap v915: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-14T17:58:47.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:46 smithi100 ceph-mon[30138]: pgmap v915: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-14T17:58:47.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:46 smithi100 ceph-mon[39557]: pgmap v915: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-14T17:58:48.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:48 smithi179 ceph-mon[33633]: osdmap e591: 8 total, 8 up, 8 in 2024-02-14T17:58:48.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:48.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:48 smithi100 ceph-mon[30138]: osdmap e591: 8 total, 8 up, 8 in 2024-02-14T17:58:48.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:48.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:48 smithi100 ceph-mon[39557]: osdmap e591: 8 total, 8 up, 8 in 2024-02-14T17:58:48.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:49.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:49 smithi179 ceph-mon[33633]: pgmap v917: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:58:49.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:49.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:49 smithi179 ceph-mon[33633]: osdmap e592: 8 total, 8 up, 8 in 2024-02-14T17:58:49.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116"}]: dispatch 2024-02-14T17:58:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:49 smithi100 ceph-mon[30138]: pgmap v917: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:58:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:49.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:49 smithi100 ceph-mon[30138]: osdmap e592: 8 total, 8 up, 8 in 2024-02-14T17:58:49.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116"}]: dispatch 2024-02-14T17:58:49.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:49 smithi100 ceph-mon[39557]: pgmap v917: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:58:49.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:49.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:49 smithi100 ceph-mon[39557]: osdmap e592: 8 total, 8 up, 8 in 2024-02-14T17:58:49.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116"}]: dispatch 2024-02-14T17:58:50.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:50 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:50.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116"}]': finished 2024-02-14T17:58:50.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:50 smithi179 ceph-mon[33633]: osdmap e593: 8 total, 8 up, 8 in 2024-02-14T17:58:50.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:50 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:50.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116"}]': finished 2024-02-14T17:58:50.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:50 smithi100 ceph-mon[30138]: osdmap e593: 8 total, 8 up, 8 in 2024-02-14T17:58:50.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:50 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:58:50.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-116"}]': finished 2024-02-14T17:58:50.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:50 smithi100 ceph-mon[39557]: osdmap e593: 8 total, 8 up, 8 in 2024-02-14T17:58:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:58:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:58:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:58:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:51 smithi179 ceph-mon[33633]: pgmap v920: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:58:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:51 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:51 smithi179 ceph-mon[33633]: osdmap e594: 8 total, 8 up, 8 in 2024-02-14T17:58:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:51 smithi100 ceph-mon[30138]: pgmap v920: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:58:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:51 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:51 smithi100 ceph-mon[30138]: osdmap e594: 8 total, 8 up, 8 in 2024-02-14T17:58:51.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:51 smithi100 ceph-mon[39557]: pgmap v920: 300 pgs: 300 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T17:58:51.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:51 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:51.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:51 smithi100 ceph-mon[39557]: osdmap e594: 8 total, 8 up, 8 in 2024-02-14T17:58:52.285 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:52 smithi179 ceph-mon[33633]: osdmap e595: 8 total, 8 up, 8 in 2024-02-14T17:58:52.285 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:52.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:52 smithi100 ceph-mon[30138]: osdmap e595: 8 total, 8 up, 8 in 2024-02-14T17:58:52.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:52.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:52 smithi100 ceph-mon[39557]: osdmap e595: 8 total, 8 up, 8 in 2024-02-14T17:58:52.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:58:52.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:58:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:58:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:53 smithi179 ceph-mon[33633]: pgmap v923: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-14T17:58:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:53 smithi179 ceph-mon[33633]: osdmap e596: 8 total, 8 up, 8 in 2024-02-14T17:58:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:53 smithi100 ceph-mon[30138]: pgmap v923: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-14T17:58:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:53 smithi100 ceph-mon[30138]: osdmap e596: 8 total, 8 up, 8 in 2024-02-14T17:58:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:53 smithi100 ceph-mon[39557]: pgmap v923: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-14T17:58:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:58:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:53 smithi100 ceph-mon[39557]: osdmap e596: 8 total, 8 up, 8 in 2024-02-14T17:58:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:54 smithi179 ceph-mon[33633]: osdmap e597: 8 total, 8 up, 8 in 2024-02-14T17:58:54.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:54.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:54.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:54 smithi179 ceph-mon[33633]: osdmap e598: 8 total, 8 up, 8 in 2024-02-14T17:58:54.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-118"}]: dispatch 2024-02-14T17:58:54.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[39557]: osdmap e597: 8 total, 8 up, 8 in 2024-02-14T17:58:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[39557]: osdmap e598: 8 total, 8 up, 8 in 2024-02-14T17:58:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-118"}]: dispatch 2024-02-14T17:58:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:58:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[30138]: osdmap e597: 8 total, 8 up, 8 in 2024-02-14T17:58:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:58:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:58:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[30138]: osdmap e598: 8 total, 8 up, 8 in 2024-02-14T17:58:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-118"}]: dispatch 2024-02-14T17:58:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:55 smithi179 ceph-mon[33633]: pgmap v927: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-14T17:58:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-118"}]': finished 2024-02-14T17:58:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:55 smithi179 ceph-mon[33633]: osdmap e599: 8 total, 8 up, 8 in 2024-02-14T17:58:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-118", "mode": "writeback"}]: dispatch 2024-02-14T17:58:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:55 smithi100 ceph-mon[39557]: pgmap v927: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-14T17:58:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-118"}]': finished 2024-02-14T17:58:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:55 smithi100 ceph-mon[39557]: osdmap e599: 8 total, 8 up, 8 in 2024-02-14T17:58:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-118", "mode": "writeback"}]: dispatch 2024-02-14T17:58:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:55 smithi100 ceph-mon[30138]: pgmap v927: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-14T17:58:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-118"}]': finished 2024-02-14T17:58:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:55 smithi100 ceph-mon[30138]: osdmap e599: 8 total, 8 up, 8 in 2024-02-14T17:58:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-118", "mode": "writeback"}]: dispatch 2024-02-14T17:58:56.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:56 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:56.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-118", "mode": "writeback"}]': finished 2024-02-14T17:58:56.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:56 smithi179 ceph-mon[33633]: osdmap e600: 8 total, 8 up, 8 in 2024-02-14T17:58:56.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:56 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-118", "mode": "writeback"}]': finished 2024-02-14T17:58:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:56 smithi100 ceph-mon[39557]: osdmap e600: 8 total, 8 up, 8 in 2024-02-14T17:58:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:56 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:58:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-118", "mode": "writeback"}]': finished 2024-02-14T17:58:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:56 smithi100 ceph-mon[30138]: osdmap e600: 8 total, 8 up, 8 in 2024-02-14T17:58:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:57 smithi179 ceph-mon[33633]: pgmap v930: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:57 smithi179 ceph-mon[33633]: osdmap e601: 8 total, 8 up, 8 in 2024-02-14T17:58:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:57 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:57 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:57.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:57 smithi100 ceph-mon[39557]: pgmap v930: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:57.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:57 smithi100 ceph-mon[39557]: osdmap e601: 8 total, 8 up, 8 in 2024-02-14T17:58:57.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:57 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:57.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:57 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:57 smithi100 ceph-mon[30138]: pgmap v930: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 689 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:58:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:57 smithi100 ceph-mon[30138]: osdmap e601: 8 total, 8 up, 8 in 2024-02-14T17:58:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:57 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:58:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:57 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:58:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:58 smithi179 ceph-mon[33633]: osdmap e602: 8 total, 8 up, 8 in 2024-02-14T17:58:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118"}]: dispatch 2024-02-14T17:58:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:58 smithi179 ceph-mon[33633]: pgmap v933: 300 pgs: 300 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:58:58.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:58 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:58.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:58.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:58 smithi100 ceph-mon[39557]: osdmap e602: 8 total, 8 up, 8 in 2024-02-14T17:58:58.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118"}]: dispatch 2024-02-14T17:58:58.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:58 smithi100 ceph-mon[39557]: pgmap v933: 300 pgs: 300 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:58:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:58 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:58:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:58:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:58 smithi100 ceph-mon[30138]: osdmap e602: 8 total, 8 up, 8 in 2024-02-14T17:58:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118"}]: dispatch 2024-02-14T17:58:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:58 smithi100 ceph-mon[30138]: pgmap v933: 300 pgs: 300 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:58:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:58 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:00.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:59 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:00.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118"}]': finished 2024-02-14T17:59:00.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:58:59 smithi179 ceph-mon[33633]: osdmap e603: 8 total, 8 up, 8 in 2024-02-14T17:59:00.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:59 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118"}]': finished 2024-02-14T17:59:00.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:58:59 smithi100 ceph-mon[30138]: osdmap e603: 8 total, 8 up, 8 in 2024-02-14T17:59:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:59 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-118"}]': finished 2024-02-14T17:59:00.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:58:59 smithi100 ceph-mon[39557]: osdmap e603: 8 total, 8 up, 8 in 2024-02-14T17:59:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:00 smithi100 ceph-mon[30138]: osdmap e604: 8 total, 8 up, 8 in 2024-02-14T17:59:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:00 smithi100 ceph-mon[30138]: pgmap v936: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:59:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:00 smithi100 ceph-mon[30138]: osdmap e605: 8 total, 8 up, 8 in 2024-02-14T17:59:00.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:00.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:59:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:59:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:00.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:00 smithi100 ceph-mon[39557]: osdmap e604: 8 total, 8 up, 8 in 2024-02-14T17:59:00.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:00 smithi100 ceph-mon[39557]: pgmap v936: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:59:00.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:00 smithi100 ceph-mon[39557]: osdmap e605: 8 total, 8 up, 8 in 2024-02-14T17:59:00.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:00 smithi179 ceph-mon[33633]: osdmap e604: 8 total, 8 up, 8 in 2024-02-14T17:59:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:00 smithi179 ceph-mon[33633]: pgmap v936: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:59:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:00 smithi179 ceph-mon[33633]: osdmap e605: 8 total, 8 up, 8 in 2024-02-14T17:59:01.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:02.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:59:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:59:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:02 smithi179 ceph-mon[33633]: osdmap e606: 8 total, 8 up, 8 in 2024-02-14T17:59:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:02 smithi179 ceph-mon[33633]: pgmap v939: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:02.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:02 smithi100 ceph-mon[39557]: osdmap e606: 8 total, 8 up, 8 in 2024-02-14T17:59:02.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:02.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:02 smithi100 ceph-mon[39557]: pgmap v939: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:02.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:02.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:02 smithi100 ceph-mon[30138]: osdmap e606: 8 total, 8 up, 8 in 2024-02-14T17:59:02.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:02.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:02 smithi100 ceph-mon[30138]: pgmap v939: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:03 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:03 smithi179 ceph-mon[33633]: osdmap e607: 8 total, 8 up, 8 in 2024-02-14T17:59:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-120"}]: dispatch 2024-02-14T17:59:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-120"}]': finished 2024-02-14T17:59:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:03 smithi179 ceph-mon[33633]: osdmap e608: 8 total, 8 up, 8 in 2024-02-14T17:59:03.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-120", "mode": "writeback"}]: dispatch 2024-02-14T17:59:03.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:03.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[30138]: osdmap e607: 8 total, 8 up, 8 in 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-120"}]: dispatch 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-120"}]': finished 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[30138]: osdmap e608: 8 total, 8 up, 8 in 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-120", "mode": "writeback"}]: dispatch 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[39557]: osdmap e607: 8 total, 8 up, 8 in 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-120"}]: dispatch 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-120"}]': finished 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[39557]: osdmap e608: 8 total, 8 up, 8 in 2024-02-14T17:59:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-120", "mode": "writeback"}]: dispatch 2024-02-14T17:59:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:04 smithi179 ceph-mon[33633]: pgmap v942: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:04 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-120", "mode": "writeback"}]': finished 2024-02-14T17:59:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:04 smithi179 ceph-mon[33633]: osdmap e609: 8 total, 8 up, 8 in 2024-02-14T17:59:04.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:04 smithi100 ceph-mon[39557]: pgmap v942: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:04.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:04 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-120", "mode": "writeback"}]': finished 2024-02-14T17:59:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:04 smithi100 ceph-mon[39557]: osdmap e609: 8 total, 8 up, 8 in 2024-02-14T17:59:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:04 smithi100 ceph-mon[30138]: pgmap v942: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:04 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-120", "mode": "writeback"}]': finished 2024-02-14T17:59:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:04 smithi100 ceph-mon[30138]: osdmap e609: 8 total, 8 up, 8 in 2024-02-14T17:59:05.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:05.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:05.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:06 smithi179 ceph-mon[33633]: osdmap e610: 8 total, 8 up, 8 in 2024-02-14T17:59:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120"}]: dispatch 2024-02-14T17:59:06.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:06 smithi179 ceph-mon[33633]: pgmap v945: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:06.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:06.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:06 smithi100 ceph-mon[30138]: osdmap e610: 8 total, 8 up, 8 in 2024-02-14T17:59:06.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120"}]: dispatch 2024-02-14T17:59:06.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:06 smithi100 ceph-mon[30138]: pgmap v945: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:06.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:06.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:06 smithi100 ceph-mon[39557]: osdmap e610: 8 total, 8 up, 8 in 2024-02-14T17:59:06.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120"}]: dispatch 2024-02-14T17:59:06.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:06 smithi100 ceph-mon[39557]: pgmap v945: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:08.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:07 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:08.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120"}]': finished 2024-02-14T17:59:08.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:07 smithi179 ceph-mon[33633]: osdmap e611: 8 total, 8 up, 8 in 2024-02-14T17:59:08.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:07 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:08.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120"}]': finished 2024-02-14T17:59:08.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:07 smithi100 ceph-mon[39557]: osdmap e611: 8 total, 8 up, 8 in 2024-02-14T17:59:08.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:07 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:08.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-120"}]': finished 2024-02-14T17:59:08.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:07 smithi100 ceph-mon[30138]: osdmap e611: 8 total, 8 up, 8 in 2024-02-14T17:59:09.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:08 smithi179 ceph-mon[33633]: osdmap e612: 8 total, 8 up, 8 in 2024-02-14T17:59:09.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:08 smithi179 ceph-mon[33633]: pgmap v948: 268 pgs: 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:59:09.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:08 smithi179 ceph-mon[33633]: osdmap e613: 8 total, 8 up, 8 in 2024-02-14T17:59:09.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:08 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:09.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:08 smithi100 ceph-mon[30138]: osdmap e612: 8 total, 8 up, 8 in 2024-02-14T17:59:09.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:08 smithi100 ceph-mon[30138]: pgmap v948: 268 pgs: 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:59:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:08 smithi100 ceph-mon[30138]: osdmap e613: 8 total, 8 up, 8 in 2024-02-14T17:59:09.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:08 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:09.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:08 smithi100 ceph-mon[39557]: osdmap e612: 8 total, 8 up, 8 in 2024-02-14T17:59:09.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:08 smithi100 ceph-mon[39557]: pgmap v948: 268 pgs: 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:59:09.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:08 smithi100 ceph-mon[39557]: osdmap e613: 8 total, 8 up, 8 in 2024-02-14T17:59:09.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:08 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:10 smithi179 ceph-mon[33633]: osdmap e614: 8 total, 8 up, 8 in 2024-02-14T17:59:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:10.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:10 smithi179 ceph-mon[33633]: pgmap v951: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:59:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:59:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:59:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:10.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:10.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:10 smithi100 ceph-mon[30138]: osdmap e614: 8 total, 8 up, 8 in 2024-02-14T17:59:10.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:10.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:10 smithi100 ceph-mon[30138]: pgmap v951: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:59:10.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:10.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:10 smithi100 ceph-mon[39557]: osdmap e614: 8 total, 8 up, 8 in 2024-02-14T17:59:10.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:10.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:10 smithi100 ceph-mon[39557]: pgmap v951: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T17:59:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:11 smithi179 ceph-mon[33633]: osdmap e615: 8 total, 8 up, 8 in 2024-02-14T17:59:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-122"}]: dispatch 2024-02-14T17:59:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:11 smithi100 ceph-mon[30138]: osdmap e615: 8 total, 8 up, 8 in 2024-02-14T17:59:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-122"}]: dispatch 2024-02-14T17:59:11.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:11.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:11 smithi100 ceph-mon[39557]: osdmap e615: 8 total, 8 up, 8 in 2024-02-14T17:59:11.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-122"}]: dispatch 2024-02-14T17:59:12.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:59:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:59:12.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-122"}]': finished 2024-02-14T17:59:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:12 smithi179 ceph-mon[33633]: osdmap e616: 8 total, 8 up, 8 in 2024-02-14T17:59:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-122", "mode": "writeback"}]: dispatch 2024-02-14T17:59:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:12 smithi179 ceph-mon[33633]: pgmap v954: 300 pgs: 300 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:12 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:12 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:12 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:12.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-122", "mode": "writeback"}]': finished 2024-02-14T17:59:12.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-122"}]': finished 2024-02-14T17:59:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[39557]: osdmap e616: 8 total, 8 up, 8 in 2024-02-14T17:59:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-122", "mode": "writeback"}]: dispatch 2024-02-14T17:59:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[39557]: pgmap v954: 300 pgs: 300 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-122", "mode": "writeback"}]': finished 2024-02-14T17:59:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-122"}]': finished 2024-02-14T17:59:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[30138]: osdmap e616: 8 total, 8 up, 8 in 2024-02-14T17:59:12.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-122", "mode": "writeback"}]: dispatch 2024-02-14T17:59:12.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[30138]: pgmap v954: 300 pgs: 300 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:12.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:12.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:12.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:12.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-122", "mode": "writeback"}]': finished 2024-02-14T17:59:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:13 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:13 smithi179 ceph-mon[33633]: osdmap e617: 8 total, 8 up, 8 in 2024-02-14T17:59:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:13.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:13 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:13.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:13 smithi100 ceph-mon[39557]: osdmap e617: 8 total, 8 up, 8 in 2024-02-14T17:59:13.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:13 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:13 smithi100 ceph-mon[30138]: osdmap e617: 8 total, 8 up, 8 in 2024-02-14T17:59:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:14.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:14 smithi179 ceph-mon[33633]: osdmap e618: 8 total, 8 up, 8 in 2024-02-14T17:59:14.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122"}]: dispatch 2024-02-14T17:59:14.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:14 smithi179 ceph-mon[33633]: pgmap v957: 300 pgs: 300 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:14.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:14 smithi100 ceph-mon[39557]: osdmap e618: 8 total, 8 up, 8 in 2024-02-14T17:59:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122"}]: dispatch 2024-02-14T17:59:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:14 smithi100 ceph-mon[39557]: pgmap v957: 300 pgs: 300 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:14.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:14.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:14 smithi100 ceph-mon[30138]: osdmap e618: 8 total, 8 up, 8 in 2024-02-14T17:59:14.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122"}]: dispatch 2024-02-14T17:59:14.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:14 smithi100 ceph-mon[30138]: pgmap v957: 300 pgs: 300 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:15 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:15 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122"}]': finished 2024-02-14T17:59:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:15 smithi179 ceph-mon[33633]: osdmap e619: 8 total, 8 up, 8 in 2024-02-14T17:59:15.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:15 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:15.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:15 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122"}]': finished 2024-02-14T17:59:15.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:15 smithi100 ceph-mon[39557]: osdmap e619: 8 total, 8 up, 8 in 2024-02-14T17:59:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:15 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:15 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-122"}]': finished 2024-02-14T17:59:15.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:15 smithi100 ceph-mon[30138]: osdmap e619: 8 total, 8 up, 8 in 2024-02-14T17:59:16.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:16 smithi100 ceph-mon[39557]: osdmap e620: 8 total, 8 up, 8 in 2024-02-14T17:59:16.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:16 smithi100 ceph-mon[39557]: pgmap v960: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:16 smithi100 ceph-mon[30138]: osdmap e620: 8 total, 8 up, 8 in 2024-02-14T17:59:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:16 smithi100 ceph-mon[30138]: pgmap v960: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:17.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:16 smithi179 ceph-mon[33633]: osdmap e620: 8 total, 8 up, 8 in 2024-02-14T17:59:17.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:16 smithi179 ceph-mon[33633]: pgmap v960: 268 pgs: 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:17.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:17 smithi100 ceph-mon[39557]: osdmap e621: 8 total, 8 up, 8 in 2024-02-14T17:59:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:17 smithi100 ceph-mon[30138]: osdmap e621: 8 total, 8 up, 8 in 2024-02-14T17:59:17.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:18.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:17 smithi179 ceph-mon[33633]: osdmap e621: 8 total, 8 up, 8 in 2024-02-14T17:59:18.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:18.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:18.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:18 smithi100 ceph-mon[39557]: osdmap e622: 8 total, 8 up, 8 in 2024-02-14T17:59:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:18 smithi100 ceph-mon[39557]: pgmap v963: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:18 smithi100 ceph-mon[30138]: osdmap e622: 8 total, 8 up, 8 in 2024-02-14T17:59:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:18 smithi100 ceph-mon[30138]: pgmap v963: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:19.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:19.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:18 smithi179 ceph-mon[33633]: osdmap e622: 8 total, 8 up, 8 in 2024-02-14T17:59:19.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:18 smithi179 ceph-mon[33633]: pgmap v963: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:19.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:19 smithi100 ceph-mon[39557]: osdmap e623: 8 total, 8 up, 8 in 2024-02-14T17:59:19.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:19.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:19 smithi100 ceph-mon[30138]: osdmap e623: 8 total, 8 up, 8 in 2024-02-14T17:59:19.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:20.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:19 smithi179 ceph-mon[33633]: osdmap e623: 8 total, 8 up, 8 in 2024-02-14T17:59:20.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:59:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:59:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:20 smithi100 ceph-mon[39557]: osdmap e624: 8 total, 8 up, 8 in 2024-02-14T17:59:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-124"}]: dispatch 2024-02-14T17:59:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:20 smithi100 ceph-mon[39557]: pgmap v966: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:20 smithi100 ceph-mon[30138]: osdmap e624: 8 total, 8 up, 8 in 2024-02-14T17:59:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-124"}]: dispatch 2024-02-14T17:59:20.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:20 smithi100 ceph-mon[30138]: pgmap v966: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:20 smithi179 ceph-mon[33633]: osdmap e624: 8 total, 8 up, 8 in 2024-02-14T17:59:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-124"}]: dispatch 2024-02-14T17:59:21.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:20 smithi179 ceph-mon[33633]: pgmap v966: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:21.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-124"}]': finished 2024-02-14T17:59:21.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[39557]: osdmap e625: 8 total, 8 up, 8 in 2024-02-14T17:59:21.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-124", "mode": "writeback"}]: dispatch 2024-02-14T17:59:21.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:21.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-124", "mode": "writeback"}]': finished 2024-02-14T17:59:21.902 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[39557]: osdmap e626: 8 total, 8 up, 8 in 2024-02-14T17:59:21.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-124"}]': finished 2024-02-14T17:59:21.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[30138]: osdmap e625: 8 total, 8 up, 8 in 2024-02-14T17:59:21.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-124", "mode": "writeback"}]: dispatch 2024-02-14T17:59:21.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:21.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-124", "mode": "writeback"}]': finished 2024-02-14T17:59:21.902 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:21 smithi100 ceph-mon[30138]: osdmap e626: 8 total, 8 up, 8 in 2024-02-14T17:59:22.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-124"}]': finished 2024-02-14T17:59:22.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:21 smithi179 ceph-mon[33633]: osdmap e625: 8 total, 8 up, 8 in 2024-02-14T17:59:22.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-124", "mode": "writeback"}]: dispatch 2024-02-14T17:59:22.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:21 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:22.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-124", "mode": "writeback"}]': finished 2024-02-14T17:59:22.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:21 smithi179 ceph-mon[33633]: osdmap e626: 8 total, 8 up, 8 in 2024-02-14T17:59:22.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:59:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:59:22.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:22 smithi100 ceph-mon[39557]: pgmap v969: 300 pgs: 300 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:59:22.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:22 smithi100 ceph-mon[30138]: pgmap v969: 300 pgs: 300 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:59:23.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:22 smithi179 ceph-mon[33633]: pgmap v969: 300 pgs: 300 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:59:23.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:23 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:23.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:23 smithi100 ceph-mon[39557]: osdmap e627: 8 total, 8 up, 8 in 2024-02-14T17:59:23.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:23.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:23 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:23.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:23 smithi100 ceph-mon[30138]: osdmap e627: 8 total, 8 up, 8 in 2024-02-14T17:59:23.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:24.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:23 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:24.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:23 smithi179 ceph-mon[33633]: osdmap e627: 8 total, 8 up, 8 in 2024-02-14T17:59:24.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[30138]: osdmap e628: 8 total, 8 up, 8 in 2024-02-14T17:59:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124"}]: dispatch 2024-02-14T17:59:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[30138]: pgmap v972: 300 pgs: 300 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:59:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:24.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124"}]': finished 2024-02-14T17:59:24.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:24.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[39557]: osdmap e628: 8 total, 8 up, 8 in 2024-02-14T17:59:24.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124"}]: dispatch 2024-02-14T17:59:24.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[39557]: pgmap v972: 300 pgs: 300 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:59:24.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:24.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124"}]': finished 2024-02-14T17:59:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:24 smithi179 ceph-mon[33633]: osdmap e628: 8 total, 8 up, 8 in 2024-02-14T17:59:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124"}]: dispatch 2024-02-14T17:59:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:24 smithi179 ceph-mon[33633]: pgmap v972: 300 pgs: 300 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-14T17:59:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:24 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-124"}]': finished 2024-02-14T17:59:25.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:25 smithi100 ceph-mon[30138]: osdmap e629: 8 total, 8 up, 8 in 2024-02-14T17:59:25.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:25 smithi100 ceph-mon[39557]: osdmap e629: 8 total, 8 up, 8 in 2024-02-14T17:59:26.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:25 smithi179 ceph-mon[33633]: osdmap e629: 8 total, 8 up, 8 in 2024-02-14T17:59:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:26 smithi179 ceph-mon[33633]: osdmap e630: 8 total, 8 up, 8 in 2024-02-14T17:59:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:26 smithi179 ceph-mon[33633]: pgmap v975: 268 pgs: 268 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:27.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:26 smithi100 ceph-mon[30138]: osdmap e630: 8 total, 8 up, 8 in 2024-02-14T17:59:27.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:26 smithi100 ceph-mon[30138]: pgmap v975: 268 pgs: 268 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:27.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:26 smithi100 ceph-mon[39557]: osdmap e630: 8 total, 8 up, 8 in 2024-02-14T17:59:27.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:26 smithi100 ceph-mon[39557]: pgmap v975: 268 pgs: 268 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:27 smithi179 ceph-mon[33633]: osdmap e631: 8 total, 8 up, 8 in 2024-02-14T17:59:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:27 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:28.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:27 smithi100 ceph-mon[39557]: osdmap e631: 8 total, 8 up, 8 in 2024-02-14T17:59:28.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:28.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:27 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:27 smithi100 ceph-mon[30138]: osdmap e631: 8 total, 8 up, 8 in 2024-02-14T17:59:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:28.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:27 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:28 smithi179 ceph-mon[33633]: osdmap e632: 8 total, 8 up, 8 in 2024-02-14T17:59:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:29.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:28 smithi179 ceph-mon[33633]: pgmap v978: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:29.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:28 smithi100 ceph-mon[39557]: osdmap e632: 8 total, 8 up, 8 in 2024-02-14T17:59:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:29.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:28 smithi100 ceph-mon[39557]: pgmap v978: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:28 smithi100 ceph-mon[30138]: osdmap e632: 8 total, 8 up, 8 in 2024-02-14T17:59:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:29.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:28 smithi100 ceph-mon[30138]: pgmap v978: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:30.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:30.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:29 smithi179 ceph-mon[33633]: osdmap e633: 8 total, 8 up, 8 in 2024-02-14T17:59:30.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-126"}]: dispatch 2024-02-14T17:59:30.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:30.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:29 smithi100 ceph-mon[39557]: osdmap e633: 8 total, 8 up, 8 in 2024-02-14T17:59:30.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-126"}]: dispatch 2024-02-14T17:59:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:29 smithi100 ceph-mon[30138]: osdmap e633: 8 total, 8 up, 8 in 2024-02-14T17:59:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-126"}]: dispatch 2024-02-14T17:59:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:59:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-126"}]': finished 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: osdmap e634: 8 total, 8 up, 8 in 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-126", "mode": "writeback"}]: dispatch 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: pgmap v981: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-126", "mode": "writeback"}]': finished 2024-02-14T17:59:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[30138]: osdmap e635: 8 total, 8 up, 8 in 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-126"}]': finished 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: osdmap e634: 8 total, 8 up, 8 in 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-126", "mode": "writeback"}]: dispatch 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: pgmap v981: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-126", "mode": "writeback"}]': finished 2024-02-14T17:59:30.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:30 smithi100 ceph-mon[39557]: osdmap e635: 8 total, 8 up, 8 in 2024-02-14T17:59:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-126"}]': finished 2024-02-14T17:59:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: osdmap e634: 8 total, 8 up, 8 in 2024-02-14T17:59:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-126", "mode": "writeback"}]: dispatch 2024-02-14T17:59:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: pgmap v981: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T17:59:31.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T17:59:31.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T17:59:31.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:31.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:31.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-126", "mode": "writeback"}]': finished 2024-02-14T17:59:31.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:30 smithi179 ceph-mon[33633]: osdmap e635: 8 total, 8 up, 8 in 2024-02-14T17:59:32.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:32.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:32.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:32.704 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:59:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:59:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:32 smithi179 ceph-mon[33633]: osdmap e636: 8 total, 8 up, 8 in 2024-02-14T17:59:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126"}]: dispatch 2024-02-14T17:59:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:32 smithi179 ceph-mon[33633]: pgmap v984: 300 pgs: 300 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:59:33.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:32 smithi100 ceph-mon[39557]: osdmap e636: 8 total, 8 up, 8 in 2024-02-14T17:59:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126"}]: dispatch 2024-02-14T17:59:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:32 smithi100 ceph-mon[39557]: pgmap v984: 300 pgs: 300 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:59:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:32 smithi100 ceph-mon[30138]: osdmap e636: 8 total, 8 up, 8 in 2024-02-14T17:59:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126"}]: dispatch 2024-02-14T17:59:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:32 smithi100 ceph-mon[30138]: pgmap v984: 300 pgs: 300 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-14T17:59:33.475 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 s api_tier_pp: require_osd_release = squid 2024-02-14T17:59:33.475 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9045 ms) 2024-02-14T17:59:33.475 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6039 ms) 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5041 ms) 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25493 ms) 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14060 ms) 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12053 ms) 2024-02-14T17:59:33.476 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 00000012 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12087 ms) 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: require_osd_release = squid 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6052 ms) 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (553720 ms total) 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1470 ms) 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10603 ms) 2024-02-14T17:59:33.477 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18133 ms) 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30208 ms total) 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7039 ms) 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8057 ms) 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: waiting for scrub... 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: done waiting 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24581 ms) 2024-02-14T17:59:33.478 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-14T17:59:33.479 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9459 ms) 2024-02-14T17:59:33.479 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-14T17:59:33.479 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8038 ms) 2024-02-14T17:59:33.479 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-14T17:59:33.479 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8056 ms) 2024-02-14T17:59:33.479 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-14T17:59:33.479 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10076 ms) 2024-02-14T17:59:33.479 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-14T17:59:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:33 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:33 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126"}]': finished 2024-02-14T17:59:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:33 smithi179 ceph-mon[33633]: osdmap e637: 8 total, 8 up, 8 in 2024-02-14T17:59:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:33 smithi179 ceph-mon[33633]: osdmap e638: 8 total, 8 up, 8 in 2024-02-14T17:59:34.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:33 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:34.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:33 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126"}]': finished 2024-02-14T17:59:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:33 smithi100 ceph-mon[39557]: osdmap e637: 8 total, 8 up, 8 in 2024-02-14T17:59:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:33 smithi100 ceph-mon[39557]: osdmap e638: 8 total, 8 up, 8 in 2024-02-14T17:59:34.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:33 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:34.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:33 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:34.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-126"}]': finished 2024-02-14T17:59:34.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:33 smithi100 ceph-mon[30138]: osdmap e637: 8 total, 8 up, 8 in 2024-02-14T17:59:34.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:33 smithi100 ceph-mon[30138]: osdmap e638: 8 total, 8 up, 8 in 2024-02-14T17:59:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:34 smithi179 ceph-mon[33633]: pgmap v987: 268 pgs: 268 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:34 smithi179 ceph-mon[33633]: osdmap e639: 8 total, 8 up, 8 in 2024-02-14T17:59:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:35.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:34 smithi100 ceph-mon[39557]: pgmap v987: 268 pgs: 268 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:35.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:34 smithi100 ceph-mon[39557]: osdmap e639: 8 total, 8 up, 8 in 2024-02-14T17:59:35.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:35.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:34 smithi100 ceph-mon[30138]: pgmap v987: 268 pgs: 268 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-14T17:59:35.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:34 smithi100 ceph-mon[30138]: osdmap e639: 8 total, 8 up, 8 in 2024-02-14T17:59:35.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:36.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:36 smithi179 ceph-mon[33633]: osdmap e640: 8 total, 8 up, 8 in 2024-02-14T17:59:36.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:36.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:36 smithi179 ceph-mon[33633]: pgmap v990: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:36.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:36.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:36 smithi100 ceph-mon[39557]: osdmap e640: 8 total, 8 up, 8 in 2024-02-14T17:59:36.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:36.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:36 smithi100 ceph-mon[39557]: pgmap v990: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:36.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:36.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:36 smithi100 ceph-mon[30138]: osdmap e640: 8 total, 8 up, 8 in 2024-02-14T17:59:36.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:36.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:36 smithi100 ceph-mon[30138]: pgmap v990: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:37.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:37.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:37 smithi179 ceph-mon[33633]: osdmap e641: 8 total, 8 up, 8 in 2024-02-14T17:59:37.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-128"}]: dispatch 2024-02-14T17:59:37.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:37.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:37 smithi100 ceph-mon[39557]: osdmap e641: 8 total, 8 up, 8 in 2024-02-14T17:59:37.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-128"}]: dispatch 2024-02-14T17:59:37.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:37.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:37 smithi100 ceph-mon[30138]: osdmap e641: 8 total, 8 up, 8 in 2024-02-14T17:59:37.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-128"}]: dispatch 2024-02-14T17:59:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-128"}]': finished 2024-02-14T17:59:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:38 smithi179 ceph-mon[33633]: osdmap e642: 8 total, 8 up, 8 in 2024-02-14T17:59:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-128", "mode": "writeback"}]: dispatch 2024-02-14T17:59:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:38 smithi179 ceph-mon[33633]: pgmap v993: 300 pgs: 300 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:38 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-128", "mode": "writeback"}]': finished 2024-02-14T17:59:38.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:38 smithi179 ceph-mon[33633]: osdmap e643: 8 total, 8 up, 8 in 2024-02-14T17:59:38.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-128"}]': finished 2024-02-14T17:59:38.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[30138]: osdmap e642: 8 total, 8 up, 8 in 2024-02-14T17:59:38.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-128", "mode": "writeback"}]: dispatch 2024-02-14T17:59:38.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[30138]: pgmap v993: 300 pgs: 300 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:38.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:38.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-128", "mode": "writeback"}]': finished 2024-02-14T17:59:38.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[30138]: osdmap e643: 8 total, 8 up, 8 in 2024-02-14T17:59:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-128"}]': finished 2024-02-14T17:59:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[39557]: osdmap e642: 8 total, 8 up, 8 in 2024-02-14T17:59:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-128", "mode": "writeback"}]: dispatch 2024-02-14T17:59:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[39557]: pgmap v993: 300 pgs: 300 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-128", "mode": "writeback"}]': finished 2024-02-14T17:59:38.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:38 smithi100 ceph-mon[39557]: osdmap e643: 8 total, 8 up, 8 in 2024-02-14T17:59:40.850 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:59:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:59:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:41.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:40 smithi100 ceph-mon[30138]: pgmap v995: 300 pgs: 300 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:41.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:40 smithi100 ceph-mon[39557]: pgmap v995: 300 pgs: 300 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:41.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:40 smithi179 ceph-mon[33633]: pgmap v995: 300 pgs: 300 active+clean; 582 KiB data, 693 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T17:59:42.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:59:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:59:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:43 smithi179 ceph-mon[33633]: pgmap v996: 300 pgs: 300 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 2 op/s 2024-02-14T17:59:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:43 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:43 smithi100 ceph-mon[39557]: pgmap v996: 300 pgs: 300 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 2 op/s 2024-02-14T17:59:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:43 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:43 smithi100 ceph-mon[30138]: pgmap v996: 300 pgs: 300 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 2 op/s 2024-02-14T17:59:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:43 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:44.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:44.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:45 smithi179 ceph-mon[33633]: pgmap v997: 300 pgs: 300 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 839 B/s rd, 279 B/s wr, 1 op/s 2024-02-14T17:59:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:45 smithi179 ceph-mon[33633]: osdmap e644: 8 total, 8 up, 8 in 2024-02-14T17:59:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128"}]: dispatch 2024-02-14T17:59:45.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:45 smithi100 ceph-mon[30138]: pgmap v997: 300 pgs: 300 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 839 B/s rd, 279 B/s wr, 1 op/s 2024-02-14T17:59:45.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:45.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:45 smithi100 ceph-mon[30138]: osdmap e644: 8 total, 8 up, 8 in 2024-02-14T17:59:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128"}]: dispatch 2024-02-14T17:59:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:45 smithi100 ceph-mon[39557]: pgmap v997: 300 pgs: 300 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 839 B/s rd, 279 B/s wr, 1 op/s 2024-02-14T17:59:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:45 smithi100 ceph-mon[39557]: osdmap e644: 8 total, 8 up, 8 in 2024-02-14T17:59:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128"}]: dispatch 2024-02-14T17:59:46.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:46 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:46.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128"}]': finished 2024-02-14T17:59:46.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:46 smithi100 ceph-mon[39557]: osdmap e645: 8 total, 8 up, 8 in 2024-02-14T17:59:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:46 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128"}]': finished 2024-02-14T17:59:46.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:46 smithi100 ceph-mon[30138]: osdmap e645: 8 total, 8 up, 8 in 2024-02-14T17:59:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:46 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-128"}]': finished 2024-02-14T17:59:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:46 smithi179 ceph-mon[33633]: osdmap e645: 8 total, 8 up, 8 in 2024-02-14T17:59:47.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:47 smithi100 ceph-mon[39557]: pgmap v1000: 300 pgs: 300 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 839 B/s rd, 279 B/s wr, 1 op/s 2024-02-14T17:59:47.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:47 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:47.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:47 smithi100 ceph-mon[39557]: osdmap e646: 8 total, 8 up, 8 in 2024-02-14T17:59:47.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:47 smithi100 ceph-mon[30138]: pgmap v1000: 300 pgs: 300 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 839 B/s rd, 279 B/s wr, 1 op/s 2024-02-14T17:59:47.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:47 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:47.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:47 smithi100 ceph-mon[30138]: osdmap e646: 8 total, 8 up, 8 in 2024-02-14T17:59:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:47 smithi179 ceph-mon[33633]: pgmap v1000: 300 pgs: 300 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 839 B/s rd, 279 B/s wr, 1 op/s 2024-02-14T17:59:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:47 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:47 smithi179 ceph-mon[33633]: osdmap e646: 8 total, 8 up, 8 in 2024-02-14T17:59:48.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:48 smithi100 ceph-mon[30138]: osdmap e647: 8 total, 8 up, 8 in 2024-02-14T17:59:48.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:48.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:48.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:48 smithi100 ceph-mon[30138]: osdmap e648: 8 total, 8 up, 8 in 2024-02-14T17:59:48.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:48.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:48 smithi100 ceph-mon[39557]: osdmap e647: 8 total, 8 up, 8 in 2024-02-14T17:59:48.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:48.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:48.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:48 smithi100 ceph-mon[39557]: osdmap e648: 8 total, 8 up, 8 in 2024-02-14T17:59:48.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:48 smithi179 ceph-mon[33633]: osdmap e647: 8 total, 8 up, 8 in 2024-02-14T17:59:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:48 smithi179 ceph-mon[33633]: osdmap e648: 8 total, 8 up, 8 in 2024-02-14T17:59:48.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:49 smithi100 ceph-mon[30138]: pgmap v1003: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:49 smithi100 ceph-mon[30138]: osdmap e649: 8 total, 8 up, 8 in 2024-02-14T17:59:49.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-130"}]: dispatch 2024-02-14T17:59:49.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:49 smithi100 ceph-mon[39557]: pgmap v1003: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:49.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:49.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:49 smithi100 ceph-mon[39557]: osdmap e649: 8 total, 8 up, 8 in 2024-02-14T17:59:49.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-130"}]: dispatch 2024-02-14T17:59:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:49 smithi179 ceph-mon[33633]: pgmap v1003: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:49 smithi179 ceph-mon[33633]: osdmap e649: 8 total, 8 up, 8 in 2024-02-14T17:59:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-130"}]: dispatch 2024-02-14T17:59:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 17:59:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:17:59:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T17:59:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:51 smithi100 ceph-mon[30138]: pgmap v1006: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-130"}]': finished 2024-02-14T17:59:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:51 smithi100 ceph-mon[30138]: osdmap e650: 8 total, 8 up, 8 in 2024-02-14T17:59:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:51 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-130", "mode": "writeback"}]: dispatch 2024-02-14T17:59:51.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:51 smithi100 ceph-mon[39557]: pgmap v1006: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:51.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-130"}]': finished 2024-02-14T17:59:51.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:51 smithi100 ceph-mon[39557]: osdmap e650: 8 total, 8 up, 8 in 2024-02-14T17:59:51.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:51 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-130", "mode": "writeback"}]: dispatch 2024-02-14T17:59:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:51 smithi179 ceph-mon[33633]: pgmap v1006: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 694 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-130"}]': finished 2024-02-14T17:59:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:51 smithi179 ceph-mon[33633]: osdmap e650: 8 total, 8 up, 8 in 2024-02-14T17:59:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:51 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-130", "mode": "writeback"}]: dispatch 2024-02-14T17:59:52.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:52 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:52.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-130", "mode": "writeback"}]': finished 2024-02-14T17:59:52.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:52 smithi100 ceph-mon[30138]: osdmap e651: 8 total, 8 up, 8 in 2024-02-14T17:59:52.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:52.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:52 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:52.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-130", "mode": "writeback"}]': finished 2024-02-14T17:59:52.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:52 smithi100 ceph-mon[39557]: osdmap e651: 8 total, 8 up, 8 in 2024-02-14T17:59:52.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:52.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 17:59:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T17:59:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:52 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T17:59:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-130", "mode": "writeback"}]': finished 2024-02-14T17:59:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:52 smithi179 ceph-mon[33633]: osdmap e651: 8 total, 8 up, 8 in 2024-02-14T17:59:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T17:59:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:53 smithi100 ceph-mon[30138]: pgmap v1009: 300 pgs: 300 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-14T17:59:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:53 smithi100 ceph-mon[30138]: osdmap e652: 8 total, 8 up, 8 in 2024-02-14T17:59:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130"}]: dispatch 2024-02-14T17:59:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:53 smithi100 ceph-mon[39557]: pgmap v1009: 300 pgs: 300 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-14T17:59:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:53 smithi100 ceph-mon[39557]: osdmap e652: 8 total, 8 up, 8 in 2024-02-14T17:59:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130"}]: dispatch 2024-02-14T17:59:53.472 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:53 smithi179 ceph-mon[33633]: pgmap v1009: 300 pgs: 300 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-14T17:59:53.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T17:59:53.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:53 smithi179 ceph-mon[33633]: osdmap e652: 8 total, 8 up, 8 in 2024-02-14T17:59:53.473 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130"}]: dispatch 2024-02-14T17:59:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:54 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130"}]': finished 2024-02-14T17:59:54.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:54 smithi100 ceph-mon[30138]: osdmap e653: 8 total, 8 up, 8 in 2024-02-14T17:59:54.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:54 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:54 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130"}]': finished 2024-02-14T17:59:54.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:54 smithi100 ceph-mon[39557]: osdmap e653: 8 total, 8 up, 8 in 2024-02-14T17:59:54.402 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:54 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T17:59:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:54 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T17:59:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-130"}]': finished 2024-02-14T17:59:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:54 smithi179 ceph-mon[33633]: osdmap e653: 8 total, 8 up, 8 in 2024-02-14T17:59:54.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:54 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T17:59:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:55 smithi100 ceph-mon[39557]: pgmap v1012: 300 pgs: 300 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-14T17:59:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:55 smithi100 ceph-mon[39557]: osdmap e654: 8 total, 8 up, 8 in 2024-02-14T17:59:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:55 smithi100 ceph-mon[30138]: pgmap v1012: 300 pgs: 300 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-14T17:59:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:55 smithi100 ceph-mon[30138]: osdmap e654: 8 total, 8 up, 8 in 2024-02-14T17:59:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:55 smithi179 ceph-mon[33633]: pgmap v1012: 300 pgs: 300 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-14T17:59:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:55 smithi179 ceph-mon[33633]: osdmap e654: 8 total, 8 up, 8 in 2024-02-14T17:59:56.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[39557]: osdmap e655: 8 total, 8 up, 8 in 2024-02-14T17:59:56.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[39557]: pgmap v1015: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[39557]: osdmap e656: 8 total, 8 up, 8 in 2024-02-14T17:59:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[30138]: osdmap e655: 8 total, 8 up, 8 in 2024-02-14T17:59:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[30138]: pgmap v1015: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[30138]: osdmap e656: 8 total, 8 up, 8 in 2024-02-14T17:59:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:56 smithi179 ceph-mon[33633]: osdmap e655: 8 total, 8 up, 8 in 2024-02-14T17:59:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T17:59:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:56 smithi179 ceph-mon[33633]: pgmap v1015: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 712 MiB used, 715 GiB / 715 GiB avail 2024-02-14T17:59:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T17:59:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:56 smithi179 ceph-mon[33633]: osdmap e656: 8 total, 8 up, 8 in 2024-02-14T17:59:56.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T17:59:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:58 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:58.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:58 smithi179 ceph-mon[33633]: osdmap e657: 8 total, 8 up, 8 in 2024-02-14T17:59:58.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]: dispatch 2024-02-14T17:59:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:58 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:58.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:58.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:58 smithi100 ceph-mon[39557]: osdmap e657: 8 total, 8 up, 8 in 2024-02-14T17:59:58.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]: dispatch 2024-02-14T17:59:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:58 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T17:59:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T17:59:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T17:59:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:58 smithi100 ceph-mon[30138]: osdmap e657: 8 total, 8 up, 8 in 2024-02-14T17:59:58.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]: dispatch 2024-02-14T17:59:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:59 smithi179 ceph-mon[33633]: pgmap v1018: 300 pgs: 300 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]': finished 2024-02-14T17:59:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:59 smithi179 ceph-mon[33633]: osdmap e658: 8 total, 8 up, 8 in 2024-02-14T17:59:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 17:59:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-132", "mode": "writeback"}]: dispatch 2024-02-14T17:59:59.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:59 smithi100 ceph-mon[39557]: pgmap v1018: 300 pgs: 300 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:59.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]': finished 2024-02-14T17:59:59.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:59 smithi100 ceph-mon[39557]: osdmap e658: 8 total, 8 up, 8 in 2024-02-14T17:59:59.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 17:59:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-132", "mode": "writeback"}]: dispatch 2024-02-14T17:59:59.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:59 smithi100 ceph-mon[30138]: pgmap v1018: 300 pgs: 300 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T17:59:59.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]': finished 2024-02-14T17:59:59.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:59 smithi100 ceph-mon[30138]: osdmap e658: 8 total, 8 up, 8 in 2024-02-14T17:59:59.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 17:59:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-132", "mode": "writeback"}]: dispatch 2024-02-14T18:00:00.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:00.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-132", "mode": "writeback"}]': finished 2024-02-14T18:00:00.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: osdmap e659: 8 total, 8 up, 8 in 2024-02-14T18:00:00.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-02-14T18:00:00.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-14T18:00:00.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: pool 'test-rados-api-smithi100-79838-132' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: application not enabled on pool 'WatchNotifysmithi100-80894-1' 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: application not enabled on pool 'poolrace.1804289383' 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: application not enabled on pool 'poolrace2.846930886' 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: application not enabled on pool 'test-rados-api-smithi100-79838-111' 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[39557]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-132", "mode": "writeback"}]': finished 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: osdmap e659: 8 total, 8 up, 8 in 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-02-14T18:00:00.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-14T18:00:00.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: pool 'test-rados-api-smithi100-79838-132' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-14T18:00:00.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-14T18:00:00.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:00:00.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: application not enabled on pool 'WatchNotifysmithi100-80894-1' 2024-02-14T18:00:00.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: application not enabled on pool 'poolrace.1804289383' 2024-02-14T18:00:00.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: application not enabled on pool 'poolrace2.846930886' 2024-02-14T18:00:00.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: application not enabled on pool 'test-rados-api-smithi100-79838-111' 2024-02-14T18:00:00.402 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-mon[30138]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:00:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-132", "mode": "writeback"}]': finished 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: osdmap e659: 8 total, 8 up, 8 in 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 5 pool(s) do not have an application enabled 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: pool 'test-rados-api-smithi100-79838-132' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: application not enabled on pool 'WatchNotifysmithi100-80894-1' 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: application not enabled on pool 'poolrace.1804289383' 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: application not enabled on pool 'poolrace2.846930886' 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: application not enabled on pool 'test-rados-api-smithi100-79838-111' 2024-02-14T18:00:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:00 smithi179 ceph-mon[33633]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:00:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:00:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:00:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:01.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:01 smithi100 ceph-mon[30138]: pgmap v1021: 300 pgs: 300 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:01.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:01 smithi100 ceph-mon[30138]: osdmap e660: 8 total, 8 up, 8 in 2024-02-14T18:00:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:01 smithi100 ceph-mon[39557]: pgmap v1021: 300 pgs: 300 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:01 smithi100 ceph-mon[39557]: osdmap e660: 8 total, 8 up, 8 in 2024-02-14T18:00:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:01 smithi179 ceph-mon[33633]: pgmap v1021: 300 pgs: 300 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:01 smithi179 ceph-mon[33633]: osdmap e660: 8 total, 8 up, 8 in 2024-02-14T18:00:02.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:02 smithi100 ceph-mon[39557]: osdmap e661: 8 total, 8 up, 8 in 2024-02-14T18:00:02.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:02.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:02 smithi100 ceph-mon[39557]: pgmap v1024: 300 pgs: 300 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T18:00:02.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:02 smithi100 ceph-mon[30138]: osdmap e661: 8 total, 8 up, 8 in 2024-02-14T18:00:02.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:02.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:02 smithi100 ceph-mon[30138]: pgmap v1024: 300 pgs: 300 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T18:00:02.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:00:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:00:02.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:02 smithi179 ceph-mon[33633]: osdmap e661: 8 total, 8 up, 8 in 2024-02-14T18:00:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:02 smithi179 ceph-mon[33633]: pgmap v1024: 300 pgs: 300 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T18:00:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:03 smithi179 ceph-mon[33633]: osdmap e662: 8 total, 8 up, 8 in 2024-02-14T18:00:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]: dispatch 2024-02-14T18:00:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:03 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:03.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:03.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:03 smithi100 ceph-mon[39557]: osdmap e662: 8 total, 8 up, 8 in 2024-02-14T18:00:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]: dispatch 2024-02-14T18:00:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:03 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:03 smithi100 ceph-mon[30138]: osdmap e662: 8 total, 8 up, 8 in 2024-02-14T18:00:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]: dispatch 2024-02-14T18:00:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:03 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]': finished 2024-02-14T18:00:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:04 smithi179 ceph-mon[33633]: osdmap e663: 8 total, 8 up, 8 in 2024-02-14T18:00:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:04 smithi179 ceph-mon[33633]: pgmap v1027: 300 pgs: 300 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T18:00:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:04 smithi179 ceph-mon[33633]: osdmap e664: 8 total, 8 up, 8 in 2024-02-14T18:00:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]': finished 2024-02-14T18:00:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:04 smithi100 ceph-mon[39557]: osdmap e663: 8 total, 8 up, 8 in 2024-02-14T18:00:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:04 smithi100 ceph-mon[39557]: pgmap v1027: 300 pgs: 300 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T18:00:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:04 smithi100 ceph-mon[39557]: osdmap e664: 8 total, 8 up, 8 in 2024-02-14T18:00:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-132"}]': finished 2024-02-14T18:00:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:04 smithi100 ceph-mon[30138]: osdmap e663: 8 total, 8 up, 8 in 2024-02-14T18:00:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:04 smithi100 ceph-mon[30138]: pgmap v1027: 300 pgs: 300 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-14T18:00:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:04 smithi100 ceph-mon[30138]: osdmap e664: 8 total, 8 up, 8 in 2024-02-14T18:00:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:05.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:06 smithi179 ceph-mon[33633]: osdmap e665: 8 total, 8 up, 8 in 2024-02-14T18:00:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132"}]: dispatch 2024-02-14T18:00:06.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:06 smithi179 ceph-mon[33633]: pgmap v1030: 300 pgs: 300 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail 2024-02-14T18:00:06.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:06.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:06 smithi100 ceph-mon[30138]: osdmap e665: 8 total, 8 up, 8 in 2024-02-14T18:00:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132"}]: dispatch 2024-02-14T18:00:06.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:06 smithi100 ceph-mon[30138]: pgmap v1030: 300 pgs: 300 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail 2024-02-14T18:00:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:06 smithi100 ceph-mon[39557]: osdmap e665: 8 total, 8 up, 8 in 2024-02-14T18:00:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132"}]: dispatch 2024-02-14T18:00:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:06 smithi100 ceph-mon[39557]: pgmap v1030: 300 pgs: 300 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail 2024-02-14T18:00:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:07 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132"}]': finished 2024-02-14T18:00:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:07 smithi179 ceph-mon[33633]: osdmap e666: 8 total, 8 up, 8 in 2024-02-14T18:00:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:07 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132"}]': finished 2024-02-14T18:00:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:07 smithi100 ceph-mon[30138]: osdmap e666: 8 total, 8 up, 8 in 2024-02-14T18:00:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:07 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-132"}]': finished 2024-02-14T18:00:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:07 smithi100 ceph-mon[39557]: osdmap e666: 8 total, 8 up, 8 in 2024-02-14T18:00:08.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:08 smithi179 ceph-mon[33633]: osdmap e667: 8 total, 8 up, 8 in 2024-02-14T18:00:08.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:08 smithi179 ceph-mon[33633]: pgmap v1033: 268 pgs: 268 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:08.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:08 smithi100 ceph-mon[39557]: osdmap e667: 8 total, 8 up, 8 in 2024-02-14T18:00:08.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:08 smithi100 ceph-mon[39557]: pgmap v1033: 268 pgs: 268 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:08.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:08 smithi100 ceph-mon[30138]: osdmap e667: 8 total, 8 up, 8 in 2024-02-14T18:00:08.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:08 smithi100 ceph-mon[30138]: pgmap v1033: 268 pgs: 268 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:09 smithi179 ceph-mon[33633]: osdmap e668: 8 total, 8 up, 8 in 2024-02-14T18:00:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:09 smithi179 ceph-mon[33633]: osdmap e669: 8 total, 8 up, 8 in 2024-02-14T18:00:09.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:09.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:09 smithi100 ceph-mon[30138]: osdmap e668: 8 total, 8 up, 8 in 2024-02-14T18:00:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:09 smithi100 ceph-mon[30138]: osdmap e669: 8 total, 8 up, 8 in 2024-02-14T18:00:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:09 smithi100 ceph-mon[39557]: osdmap e668: 8 total, 8 up, 8 in 2024-02-14T18:00:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:09 smithi100 ceph-mon[39557]: osdmap e669: 8 total, 8 up, 8 in 2024-02-14T18:00:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:10 smithi179 ceph-mon[33633]: osdmap e670: 8 total, 8 up, 8 in 2024-02-14T18:00:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-134"}]: dispatch 2024-02-14T18:00:10.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:10 smithi179 ceph-mon[33633]: pgmap v1037: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T18:00:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:00:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:00:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:10.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:10.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:10 smithi100 ceph-mon[30138]: osdmap e670: 8 total, 8 up, 8 in 2024-02-14T18:00:10.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-134"}]: dispatch 2024-02-14T18:00:10.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:10 smithi100 ceph-mon[30138]: pgmap v1037: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T18:00:10.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:10.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:10 smithi100 ceph-mon[39557]: osdmap e670: 8 total, 8 up, 8 in 2024-02-14T18:00:10.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-134"}]: dispatch 2024-02-14T18:00:10.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:10 smithi100 ceph-mon[39557]: pgmap v1037: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T18:00:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-134"}]': finished 2024-02-14T18:00:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:11 smithi179 ceph-mon[33633]: osdmap e671: 8 total, 8 up, 8 in 2024-02-14T18:00:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-134", "mode": "writeback"}]: dispatch 2024-02-14T18:00:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:11 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:11.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:11 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-134", "mode": "writeback"}]': finished 2024-02-14T18:00:11.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:11 smithi179 ceph-mon[33633]: osdmap e672: 8 total, 8 up, 8 in 2024-02-14T18:00:11.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-134"}]': finished 2024-02-14T18:00:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[30138]: osdmap e671: 8 total, 8 up, 8 in 2024-02-14T18:00:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-134", "mode": "writeback"}]: dispatch 2024-02-14T18:00:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-134", "mode": "writeback"}]': finished 2024-02-14T18:00:11.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[30138]: osdmap e672: 8 total, 8 up, 8 in 2024-02-14T18:00:11.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-134"}]': finished 2024-02-14T18:00:11.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[39557]: osdmap e671: 8 total, 8 up, 8 in 2024-02-14T18:00:11.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-134", "mode": "writeback"}]: dispatch 2024-02-14T18:00:11.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:11.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-134", "mode": "writeback"}]': finished 2024-02-14T18:00:11.901 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:11 smithi100 ceph-mon[39557]: osdmap e672: 8 total, 8 up, 8 in 2024-02-14T18:00:12.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:00:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:00:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:12 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:12 smithi179 ceph-mon[33633]: pgmap v1040: 300 pgs: 300 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T18:00:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:12 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:12.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:12 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:12.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:12 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:12.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:12 smithi100 ceph-mon[30138]: pgmap v1040: 300 pgs: 300 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T18:00:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:12 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:12.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:12 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:12 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:12 smithi100 ceph-mon[39557]: pgmap v1040: 300 pgs: 300 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-14T18:00:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:12 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:12.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:12 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:13 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:13 smithi179 ceph-mon[33633]: osdmap e673: 8 total, 8 up, 8 in 2024-02-14T18:00:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:13 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134"}]: dispatch 2024-02-14T18:00:13.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:13 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:13.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:13.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:13 smithi100 ceph-mon[39557]: osdmap e673: 8 total, 8 up, 8 in 2024-02-14T18:00:13.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:13 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134"}]: dispatch 2024-02-14T18:00:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:13 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:13 smithi100 ceph-mon[30138]: osdmap e673: 8 total, 8 up, 8 in 2024-02-14T18:00:13.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:13 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134"}]: dispatch 2024-02-14T18:00:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:14 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:14 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134"}]': finished 2024-02-14T18:00:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:14 smithi179 ceph-mon[33633]: osdmap e674: 8 total, 8 up, 8 in 2024-02-14T18:00:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:14 smithi179 ceph-mon[33633]: pgmap v1043: 300 pgs: 300 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:14.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:14 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:14.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:14 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134"}]': finished 2024-02-14T18:00:14.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:14 smithi100 ceph-mon[30138]: osdmap e674: 8 total, 8 up, 8 in 2024-02-14T18:00:14.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:14 smithi100 ceph-mon[30138]: pgmap v1043: 300 pgs: 300 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:14 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:14 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-134"}]': finished 2024-02-14T18:00:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:14 smithi100 ceph-mon[39557]: osdmap e674: 8 total, 8 up, 8 in 2024-02-14T18:00:14.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:14 smithi100 ceph-mon[39557]: pgmap v1043: 300 pgs: 300 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:15 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:15.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:15 smithi179 ceph-mon[33633]: osdmap e675: 8 total, 8 up, 8 in 2024-02-14T18:00:15.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:15 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:15.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:15 smithi100 ceph-mon[30138]: osdmap e675: 8 total, 8 up, 8 in 2024-02-14T18:00:15.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:15 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:15.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:15 smithi100 ceph-mon[39557]: osdmap e675: 8 total, 8 up, 8 in 2024-02-14T18:00:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:16 smithi179 ceph-mon[33633]: osdmap e676: 8 total, 8 up, 8 in 2024-02-14T18:00:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:16 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:16 smithi179 ceph-mon[33633]: pgmap v1046: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail 2024-02-14T18:00:16.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:16 smithi100 ceph-mon[30138]: osdmap e676: 8 total, 8 up, 8 in 2024-02-14T18:00:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:16 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:16.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:16 smithi100 ceph-mon[30138]: pgmap v1046: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail 2024-02-14T18:00:16.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:16 smithi100 ceph-mon[39557]: osdmap e676: 8 total, 8 up, 8 in 2024-02-14T18:00:16.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:16 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:16.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:16 smithi100 ceph-mon[39557]: pgmap v1046: 300 pgs: 32 unknown, 268 active+clean; 582 KiB data, 719 MiB used, 715 GiB / 715 GiB avail 2024-02-14T18:00:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:17 smithi179 ceph-mon[33633]: osdmap e677: 8 total, 8 up, 8 in 2024-02-14T18:00:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:17 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:17.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:17.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:17 smithi100 ceph-mon[30138]: osdmap e677: 8 total, 8 up, 8 in 2024-02-14T18:00:17.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:17 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:17 smithi100 ceph-mon[39557]: osdmap e677: 8 total, 8 up, 8 in 2024-02-14T18:00:17.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:17 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:18 smithi179 ceph-mon[33633]: osdmap e678: 8 total, 8 up, 8 in 2024-02-14T18:00:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:18 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-136"}]: dispatch 2024-02-14T18:00:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:18 smithi179 ceph-mon[33633]: pgmap v1049: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T18:00:18.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:18 smithi100 ceph-mon[39557]: osdmap e678: 8 total, 8 up, 8 in 2024-02-14T18:00:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:18 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-136"}]: dispatch 2024-02-14T18:00:18.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:18 smithi100 ceph-mon[39557]: pgmap v1049: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T18:00:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:18 smithi100 ceph-mon[30138]: osdmap e678: 8 total, 8 up, 8 in 2024-02-14T18:00:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:18 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-136"}]: dispatch 2024-02-14T18:00:18.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:18 smithi100 ceph-mon[30138]: pgmap v1049: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T18:00:19.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-136"}]': finished 2024-02-14T18:00:19.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:19 smithi179 ceph-mon[33633]: osdmap e679: 8 total, 8 up, 8 in 2024-02-14T18:00:19.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-136", "mode": "writeback"}]: dispatch 2024-02-14T18:00:19.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-136"}]': finished 2024-02-14T18:00:19.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:19 smithi100 ceph-mon[30138]: osdmap e679: 8 total, 8 up, 8 in 2024-02-14T18:00:19.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-136", "mode": "writeback"}]: dispatch 2024-02-14T18:00:19.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-136"}]': finished 2024-02-14T18:00:19.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:19 smithi100 ceph-mon[39557]: osdmap e679: 8 total, 8 up, 8 in 2024-02-14T18:00:19.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-136", "mode": "writeback"}]: dispatch 2024-02-14T18:00:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:20 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-136", "mode": "writeback"}]': finished 2024-02-14T18:00:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:20 smithi179 ceph-mon[33633]: osdmap e680: 8 total, 8 up, 8 in 2024-02-14T18:00:20.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:20.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:20 smithi179 ceph-mon[33633]: pgmap v1052: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T18:00:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:00:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-136", "mode": "writeback"}]': finished 2024-02-14T18:00:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-mon[30138]: osdmap e680: 8 total, 8 up, 8 in 2024-02-14T18:00:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:20.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-mon[30138]: pgmap v1052: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T18:00:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-136", "mode": "writeback"}]': finished 2024-02-14T18:00:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-mon[39557]: osdmap e680: 8 total, 8 up, 8 in 2024-02-14T18:00:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:20.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:20 smithi100 ceph-mon[39557]: pgmap v1052: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-14T18:00:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:21 smithi179 ceph-mon[33633]: osdmap e681: 8 total, 8 up, 8 in 2024-02-14T18:00:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136"}]: dispatch 2024-02-14T18:00:21.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:21 smithi100 ceph-mon[30138]: osdmap e681: 8 total, 8 up, 8 in 2024-02-14T18:00:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136"}]: dispatch 2024-02-14T18:00:21.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:00:21.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:21 smithi100 ceph-mon[39557]: osdmap e681: 8 total, 8 up, 8 in 2024-02-14T18:00:21.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136"}]: dispatch 2024-02-14T18:00:22.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:00:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:00:22.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:22 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:22.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136"}]': finished 2024-02-14T18:00:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:22 smithi179 ceph-mon[33633]: osdmap e682: 8 total, 8 up, 8 in 2024-02-14T18:00:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:22 smithi179 ceph-mon[33633]: pgmap v1055: 300 pgs: 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-14T18:00:22.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:22 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:22.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136"}]': finished 2024-02-14T18:00:22.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:22 smithi100 ceph-mon[30138]: osdmap e682: 8 total, 8 up, 8 in 2024-02-14T18:00:22.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:22 smithi100 ceph-mon[30138]: pgmap v1055: 300 pgs: 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-14T18:00:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:22 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-136"}]': finished 2024-02-14T18:00:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:22 smithi100 ceph-mon[39557]: osdmap e682: 8 total, 8 up, 8 in 2024-02-14T18:00:22.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:22 smithi100 ceph-mon[39557]: pgmap v1055: 300 pgs: 300 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-14T18:00:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:23 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:23 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:23.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:23 smithi179 ceph-mon[33633]: osdmap e683: 8 total, 8 up, 8 in 2024-02-14T18:00:23.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:23 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:23.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:23 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:23.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:23 smithi100 ceph-mon[30138]: osdmap e683: 8 total, 8 up, 8 in 2024-02-14T18:00:23.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:23 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:23.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:23 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:23.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:23 smithi100 ceph-mon[39557]: osdmap e683: 8 total, 8 up, 8 in 2024-02-14T18:00:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:24 smithi100 ceph-mon[30138]: osdmap e684: 8 total, 8 up, 8 in 2024-02-14T18:00:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:24 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:24.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:24 smithi100 ceph-mon[30138]: pgmap v1058: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-14T18:00:24.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:24 smithi100 ceph-mon[39557]: osdmap e684: 8 total, 8 up, 8 in 2024-02-14T18:00:24.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:24 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:24.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:24 smithi100 ceph-mon[39557]: pgmap v1058: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-14T18:00:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:24 smithi179 ceph-mon[33633]: osdmap e684: 8 total, 8 up, 8 in 2024-02-14T18:00:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:24 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:24 smithi179 ceph-mon[33633]: pgmap v1058: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-14T18:00:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ]ec 2024-02-14T18:00:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.553 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.554 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.555 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.556 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.557 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.558 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.559 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.560 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.561 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.562 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.563 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:00:25.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:25.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:25 smithi100 ceph-mon[39557]: osdmap e685: 8 total, 8 up, 8 in 2024-02-14T18:00:25.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:25.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:25.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:25 smithi100 ceph-mon[30138]: osdmap e685: 8 total, 8 up, 8 in 2024-02-14T18:00:25.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:26.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:26.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:25 smithi179 ceph-mon[33633]: osdmap e685: 8 total, 8 up, 8 in 2024-02-14T18:00:26.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:26.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:26.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:26 smithi100 ceph-mon[30138]: osdmap e686: 8 total, 8 up, 8 in 2024-02-14T18:00:26.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T18:00:26.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:26 smithi100 ceph-mon[30138]: pgmap v1061: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail 2024-02-14T18:00:26.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_count","val": "2"}]': finished 2024-02-14T18:00:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:26 smithi100 ceph-mon[39557]: osdmap e686: 8 total, 8 up, 8 in 2024-02-14T18:00:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T18:00:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:26 smithi100 ceph-mon[39557]: pgmap v1061: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail 2024-02-14T18:00:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_count","val": "2"}]': finished 2024-02-14T18:00:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:26 smithi179 ceph-mon[33633]: osdmap e686: 8 total, 8 up, 8 in 2024-02-14T18:00:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T18:00:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:26 smithi179 ceph-mon[33633]: pgmap v1061: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 734 MiB used, 715 GiB / 715 GiB avail 2024-02-14T18:00:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_count","val": "2"}]': finished 2024-02-14T18:00:27.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[39557]: osdmap e687: 8 total, 8 up, 8 in 2024-02-14T18:00:27.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T18:00:27.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:27.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:27.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_period","val": "600"}]': finished 2024-02-14T18:00:27.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[39557]: osdmap e688: 8 total, 8 up, 8 in 2024-02-14T18:00:27.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T18:00:27.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[30138]: osdmap e687: 8 total, 8 up, 8 in 2024-02-14T18:00:27.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T18:00:27.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:27.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:27.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_period","val": "600"}]': finished 2024-02-14T18:00:27.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[30138]: osdmap e688: 8 total, 8 up, 8 in 2024-02-14T18:00:27.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T18:00:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:27 smithi179 ceph-mon[33633]: osdmap e687: 8 total, 8 up, 8 in 2024-02-14T18:00:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T18:00:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:27 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:27 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_period","val": "600"}]': finished 2024-02-14T18:00:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:27 smithi179 ceph-mon[33633]: osdmap e688: 8 total, 8 up, 8 in 2024-02-14T18:00:28.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-14T18:00:28.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:28 smithi100 ceph-mon[39557]: pgmap v1064: 300 pgs: 300 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:28.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:28 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:28.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T18:00:28.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:28 smithi100 ceph-mon[39557]: osdmap e689: 8 total, 8 up, 8 in 2024-02-14T18:00:28.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:28 smithi100 ceph-mon[30138]: pgmap v1064: 300 pgs: 300 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:28.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:28 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:28.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T18:00:28.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:28 smithi100 ceph-mon[30138]: osdmap e689: 8 total, 8 up, 8 in 2024-02-14T18:00:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:28 smithi179 ceph-mon[33633]: pgmap v1064: 300 pgs: 300 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:28 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-14T18:00:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:28 smithi179 ceph-mon[33633]: osdmap e689: 8 total, 8 up, 8 in 2024-02-14T18:00:30.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:30.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138"}]: dispatch 2024-02-14T18:00:30.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:30.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138"}]: dispatch 2024-02-14T18:00:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:30.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138"}]: dispatch 2024-02-14T18:00:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:00:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:00:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138"}]': finished 2024-02-14T18:00:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:30 smithi100 ceph-mon[30138]: osdmap e690: 8 total, 8 up, 8 in 2024-02-14T18:00:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:30 smithi100 ceph-mon[30138]: pgmap v1067: 300 pgs: 300 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:30 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:00:30.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138"}]': finished 2024-02-14T18:00:30.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:30 smithi100 ceph-mon[39557]: osdmap e690: 8 total, 8 up, 8 in 2024-02-14T18:00:30.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:30 smithi100 ceph-mon[39557]: pgmap v1067: 300 pgs: 300 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:30.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:30 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:00:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-138"}]': finished 2024-02-14T18:00:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:30 smithi179 ceph-mon[33633]: osdmap e690: 8 total, 8 up, 8 in 2024-02-14T18:00:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:30 smithi179 ceph-mon[33633]: pgmap v1067: 300 pgs: 300 active+clean; 4.4 MiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:30 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:00:32.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:31 smithi179 ceph-mon[33633]: osdmap e691: 8 total, 8 up, 8 in 2024-02-14T18:00:32.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:31 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:32.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:31 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:32.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:31 smithi100 ceph-mon[30138]: osdmap e691: 8 total, 8 up, 8 in 2024-02-14T18:00:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:31 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:32.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:31 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:32.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:31 smithi100 ceph-mon[39557]: osdmap e691: 8 total, 8 up, 8 in 2024-02-14T18:00:32.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:31 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:32.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:31 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:32.689 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:00:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:00:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:32 smithi179 ceph-mon[33633]: osdmap e692: 8 total, 8 up, 8 in 2024-02-14T18:00:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:32 smithi179 ceph-mon[33633]: pgmap v1070: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:32 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:00:33.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:32 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:00:33.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:32 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[39557]: osdmap e692: 8 total, 8 up, 8 in 2024-02-14T18:00:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[39557]: pgmap v1070: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:00:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:00:33.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[30138]: osdmap e692: 8 total, 8 up, 8 in 2024-02-14T18:00:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[30138]: pgmap v1070: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:33.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:00:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:00:33.151 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:32 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:33 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:33 smithi179 ceph-mon[33633]: osdmap e693: 8 total, 8 up, 8 in 2024-02-14T18:00:34.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:34.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:33 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:34.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:33 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:34.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:34.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:33 smithi100 ceph-mon[30138]: osdmap e693: 8 total, 8 up, 8 in 2024-02-14T18:00:34.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:34.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:33 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:33 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:33 smithi100 ceph-mon[39557]: osdmap e693: 8 total, 8 up, 8 in 2024-02-14T18:00:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:34.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:33 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:34 smithi179 ceph-mon[33633]: osdmap e694: 8 total, 8 up, 8 in 2024-02-14T18:00:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:34 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T18:00:35.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:34 smithi179 ceph-mon[33633]: pgmap v1073: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:35.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:35.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:34 smithi100 ceph-mon[30138]: osdmap e694: 8 total, 8 up, 8 in 2024-02-14T18:00:35.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:34 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T18:00:35.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:34 smithi100 ceph-mon[30138]: pgmap v1073: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:35.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:35.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:34 smithi100 ceph-mon[39557]: osdmap e694: 8 total, 8 up, 8 in 2024-02-14T18:00:35.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:34 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-14T18:00:35.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:34 smithi100 ceph-mon[39557]: pgmap v1073: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:36.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_count","val": "3"}]': finished 2024-02-14T18:00:36.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:35 smithi179 ceph-mon[33633]: osdmap e695: 8 total, 8 up, 8 in 2024-02-14T18:00:36.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:35 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T18:00:36.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_count","val": "3"}]': finished 2024-02-14T18:00:36.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:35 smithi100 ceph-mon[30138]: osdmap e695: 8 total, 8 up, 8 in 2024-02-14T18:00:36.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:35 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T18:00:36.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_count","val": "3"}]': finished 2024-02-14T18:00:36.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:35 smithi100 ceph-mon[39557]: osdmap e695: 8 total, 8 up, 8 in 2024-02-14T18:00:36.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:35 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-14T18:00:37.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_period","val": "3"}]': finished 2024-02-14T18:00:37.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:36 smithi179 ceph-mon[33633]: osdmap e696: 8 total, 8 up, 8 in 2024-02-14T18:00:37.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T18:00:37.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:36 smithi179 ceph-mon[33633]: pgmap v1076: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:00:37.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T18:00:37.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:36 smithi179 ceph-mon[33633]: osdmap e697: 8 total, 8 up, 8 in 2024-02-14T18:00:37.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T18:00:37.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_period","val": "3"}]': finished 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[30138]: osdmap e696: 8 total, 8 up, 8 in 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[30138]: pgmap v1076: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[30138]: osdmap e697: 8 total, 8 up, 8 in 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_period","val": "3"}]': finished 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[39557]: osdmap e696: 8 total, 8 up, 8 in 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[39557]: pgmap v1076: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 757 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T18:00:37.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[39557]: osdmap e697: 8 total, 8 up, 8 in 2024-02-14T18:00:37.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-14T18:00:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T18:00:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:38 smithi179 ceph-mon[33633]: osdmap e698: 8 total, 8 up, 8 in 2024-02-14T18:00:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:38 smithi179 ceph-mon[33633]: pgmap v1079: 300 pgs: 300 active+clean; 4.4 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:39.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:38 smithi179 ceph-mon[33633]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:39.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T18:00:39.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:38 smithi100 ceph-mon[39557]: osdmap e698: 8 total, 8 up, 8 in 2024-02-14T18:00:39.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:38 smithi100 ceph-mon[39557]: pgmap v1079: 300 pgs: 300 active+clean; 4.4 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:39.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:38 smithi100 ceph-mon[39557]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-14T18:00:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:38 smithi100 ceph-mon[30138]: osdmap e698: 8 total, 8 up, 8 in 2024-02-14T18:00:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:38 smithi100 ceph-mon[30138]: pgmap v1079: 300 pgs: 300 active+clean; 4.4 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:39.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:38 smithi100 ceph-mon[30138]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:00:40.870 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:00:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:00:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:41.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:40 smithi100 ceph-mon[30138]: pgmap v1080: 300 pgs: 300 active+clean; 4.4 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1001 B/s rd, 0 op/s 2024-02-14T18:00:41.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:40 smithi100 ceph-mon[39557]: pgmap v1080: 300 pgs: 300 active+clean; 4.4 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1001 B/s rd, 0 op/s 2024-02-14T18:00:41.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:40 smithi179 ceph-mon[33633]: pgmap v1080: 300 pgs: 300 active+clean; 4.4 MiB data, 779 MiB used, 714 GiB / 715 GiB avail; 1001 B/s rd, 0 op/s 2024-02-14T18:00:42.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:00:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:00:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:42 smithi100 ceph-mon[30138]: pgmap v1081: 300 pgs: 300 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 5.4 KiB/s wr, 2 op/s 2024-02-14T18:00:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:42 smithi100 ceph-mon[39557]: pgmap v1081: 300 pgs: 300 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 5.4 KiB/s wr, 2 op/s 2024-02-14T18:00:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:42 smithi179 ceph-mon[33633]: pgmap v1081: 300 pgs: 300 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 5.4 KiB/s wr, 2 op/s 2024-02-14T18:00:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:44.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:43 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:44.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:43 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:44.324 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:43 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:45.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:44 smithi100 ceph-mon[30138]: pgmap v1082: 300 pgs: 300 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-14T18:00:45.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:44 smithi100 ceph-mon[39557]: pgmap v1082: 300 pgs: 300 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-14T18:00:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:44 smithi179 ceph-mon[33633]: pgmap v1082: 300 pgs: 300 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-14T18:00:47.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:46 smithi100 ceph-mon[30138]: pgmap v1083: 300 pgs: 300 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 561 B/s rd, 3.6 KiB/s wr, 1 op/s 2024-02-14T18:00:47.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:46 smithi100 ceph-mon[39557]: pgmap v1083: 300 pgs: 300 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 561 B/s rd, 3.6 KiB/s wr, 1 op/s 2024-02-14T18:00:47.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:46 smithi179 ceph-mon[33633]: pgmap v1083: 300 pgs: 300 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 561 B/s rd, 3.6 KiB/s wr, 1 op/s 2024-02-14T18:00:49.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:48 smithi100 ceph-mon[30138]: pgmap v1084: 300 pgs: 300 active+clean; 4.5 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1012 B/s rd, 7.3 KiB/s wr, 2 op/s 2024-02-14T18:00:49.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:48 smithi100 ceph-mon[39557]: pgmap v1084: 300 pgs: 300 active+clean; 4.5 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1012 B/s rd, 7.3 KiB/s wr, 2 op/s 2024-02-14T18:00:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:48 smithi179 ceph-mon[33633]: pgmap v1084: 300 pgs: 300 active+clean; 4.5 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1012 B/s rd, 7.3 KiB/s wr, 2 op/s 2024-02-14T18:00:50.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:50.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140"}]: dispatch 2024-02-14T18:00:50.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:50.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140"}]: dispatch 2024-02-14T18:00:50.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:00:50.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140"}]: dispatch 2024-02-14T18:00:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:00:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:00:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:00:51.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:50 smithi100 ceph-mon[30138]: pgmap v1085: 300 pgs: 300 active+clean; 4.5 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 6.2 KiB/s wr, 1 op/s 2024-02-14T18:00:51.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140"}]': finished 2024-02-14T18:00:51.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:50 smithi100 ceph-mon[30138]: osdmap e699: 8 total, 8 up, 8 in 2024-02-14T18:00:51.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:50 smithi100 ceph-mon[39557]: pgmap v1085: 300 pgs: 300 active+clean; 4.5 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 6.2 KiB/s wr, 1 op/s 2024-02-14T18:00:51.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140"}]': finished 2024-02-14T18:00:51.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:50 smithi100 ceph-mon[39557]: osdmap e699: 8 total, 8 up, 8 in 2024-02-14T18:00:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:50 smithi179 ceph-mon[33633]: pgmap v1085: 300 pgs: 300 active+clean; 4.5 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 6.2 KiB/s wr, 1 op/s 2024-02-14T18:00:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-140"}]': finished 2024-02-14T18:00:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:50 smithi179 ceph-mon[33633]: osdmap e699: 8 total, 8 up, 8 in 2024-02-14T18:00:52.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:51 smithi179 ceph-mon[33633]: osdmap e700: 8 total, 8 up, 8 in 2024-02-14T18:00:52.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:51 smithi100 ceph-mon[39557]: osdmap e700: 8 total, 8 up, 8 in 2024-02-14T18:00:52.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:51 smithi100 ceph-mon[30138]: osdmap e700: 8 total, 8 up, 8 in 2024-02-14T18:00:52.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:00:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:00:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:52 smithi179 ceph-mon[33633]: pgmap v1088: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:52 smithi179 ceph-mon[33633]: osdmap e701: 8 total, 8 up, 8 in 2024-02-14T18:00:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:52 smithi100 ceph-mon[30138]: pgmap v1088: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:53.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:52 smithi100 ceph-mon[30138]: osdmap e701: 8 total, 8 up, 8 in 2024-02-14T18:00:53.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:52 smithi100 ceph-mon[39557]: pgmap v1088: 268 pgs: 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:52 smithi100 ceph-mon[39557]: osdmap e701: 8 total, 8 up, 8 in 2024-02-14T18:00:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:00:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:53 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:53 smithi179 ceph-mon[33633]: osdmap e702: 8 total, 8 up, 8 in 2024-02-14T18:00:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:53 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:53 smithi100 ceph-mon[30138]: osdmap e702: 8 total, 8 up, 8 in 2024-02-14T18:00:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:53 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:00:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:00:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:53 smithi100 ceph-mon[39557]: osdmap e702: 8 total, 8 up, 8 in 2024-02-14T18:00:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:00:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:54 smithi179 ceph-mon[33633]: pgmap v1091: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:54 smithi179 ceph-mon[33633]: osdmap e703: 8 total, 8 up, 8 in 2024-02-14T18:00:55.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:54 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-142"}]: dispatch 2024-02-14T18:00:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:54 smithi100 ceph-mon[39557]: pgmap v1091: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:54 smithi100 ceph-mon[39557]: osdmap e703: 8 total, 8 up, 8 in 2024-02-14T18:00:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:54 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-142"}]: dispatch 2024-02-14T18:00:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:54 smithi100 ceph-mon[30138]: pgmap v1091: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:00:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:00:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:54 smithi100 ceph-mon[30138]: osdmap e703: 8 total, 8 up, 8 in 2024-02-14T18:00:55.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:54 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-142"}]: dispatch 2024-02-14T18:00:56.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-142"}]': finished 2024-02-14T18:00:56.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:55 smithi179 ceph-mon[33633]: osdmap e704: 8 total, 8 up, 8 in 2024-02-14T18:00:56.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-142", "mode": "writeback"}]: dispatch 2024-02-14T18:00:56.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:55 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:56.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-142", "mode": "writeback"}]': finished 2024-02-14T18:00:56.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:55 smithi179 ceph-mon[33633]: osdmap e705: 8 total, 8 up, 8 in 2024-02-14T18:00:56.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T18:00:56.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-142"}]': finished 2024-02-14T18:00:56.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[30138]: osdmap e704: 8 total, 8 up, 8 in 2024-02-14T18:00:56.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-142", "mode": "writeback"}]: dispatch 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-142", "mode": "writeback"}]': finished 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[30138]: osdmap e705: 8 total, 8 up, 8 in 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-142"}]': finished 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[39557]: osdmap e704: 8 total, 8 up, 8 in 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-142", "mode": "writeback"}]: dispatch 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-142", "mode": "writeback"}]': finished 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[39557]: osdmap e705: 8 total, 8 up, 8 in 2024-02-14T18:00:56.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T18:00:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:56 smithi179 ceph-mon[33633]: pgmap v1094: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:00:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_count","val": "2"}]': finished 2024-02-14T18:00:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:56 smithi179 ceph-mon[33633]: osdmap e706: 8 total, 8 up, 8 in 2024-02-14T18:00:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T18:00:57.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:56 smithi100 ceph-mon[30138]: pgmap v1094: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:00:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_count","val": "2"}]': finished 2024-02-14T18:00:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:56 smithi100 ceph-mon[30138]: osdmap e706: 8 total, 8 up, 8 in 2024-02-14T18:00:57.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T18:00:57.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:56 smithi100 ceph-mon[39557]: pgmap v1094: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 780 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:00:57.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_count","val": "2"}]': finished 2024-02-14T18:00:57.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:56 smithi100 ceph-mon[39557]: osdmap e706: 8 total, 8 up, 8 in 2024-02-14T18:00:57.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T18:00:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:58.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:58.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[39557]: pgmap v1097: 300 pgs: 300 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T18:00:58.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_period","val": "600"}]': finished 2024-02-14T18:00:58.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[39557]: osdmap e707: 8 total, 8 up, 8 in 2024-02-14T18:00:58.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T18:00:58.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:58.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:58.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[30138]: pgmap v1097: 300 pgs: 300 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T18:00:58.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_period","val": "600"}]': finished 2024-02-14T18:00:58.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[30138]: osdmap e707: 8 total, 8 up, 8 in 2024-02-14T18:00:58.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T18:00:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:58 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:00:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:00:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:58 smithi179 ceph-mon[33633]: pgmap v1097: 300 pgs: 300 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T18:00:58.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_period","val": "600"}]': finished 2024-02-14T18:00:58.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:58 smithi179 ceph-mon[33633]: osdmap e707: 8 total, 8 up, 8 in 2024-02-14T18:00:58.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T18:00:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:59 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T18:00:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:59 smithi179 ceph-mon[33633]: osdmap e708: 8 total, 8 up, 8 in 2024-02-14T18:00:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:00:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T18:00:59.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:59 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T18:00:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:59 smithi100 ceph-mon[39557]: osdmap e708: 8 total, 8 up, 8 in 2024-02-14T18:00:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:00:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T18:00:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:59 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:00:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T18:00:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:59 smithi100 ceph-mon[30138]: osdmap e708: 8 total, 8 up, 8 in 2024-02-14T18:00:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:00:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T18:01:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:01:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:01:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:00 smithi179 ceph-mon[33633]: pgmap v1100: 300 pgs: 300 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T18:01:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T18:01:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:00 smithi179 ceph-mon[33633]: osdmap e709: 8 total, 8 up, 8 in 2024-02-14T18:01:01.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T18:01:01.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:00 smithi100 ceph-mon[39557]: pgmap v1100: 300 pgs: 300 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T18:01:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T18:01:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:00 smithi100 ceph-mon[39557]: osdmap e709: 8 total, 8 up, 8 in 2024-02-14T18:01:01.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T18:01:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:00 smithi100 ceph-mon[30138]: pgmap v1100: 300 pgs: 300 active+clean; 4.4 MiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-14T18:01:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T18:01:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:00 smithi100 ceph-mon[30138]: osdmap e709: 8 total, 8 up, 8 in 2024-02-14T18:01:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-14T18:01:02.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T18:01:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:01 smithi100 ceph-mon[30138]: osdmap e710: 8 total, 8 up, 8 in 2024-02-14T18:01:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T18:01:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T18:01:02.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:01 smithi100 ceph-mon[30138]: osdmap e711: 8 total, 8 up, 8 in 2024-02-14T18:01:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T18:01:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:01 smithi100 ceph-mon[39557]: osdmap e710: 8 total, 8 up, 8 in 2024-02-14T18:01:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T18:01:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T18:01:02.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:01 smithi100 ceph-mon[39557]: osdmap e711: 8 total, 8 up, 8 in 2024-02-14T18:01:02.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-14T18:01:02.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:01 smithi179 ceph-mon[33633]: osdmap e710: 8 total, 8 up, 8 in 2024-02-14T18:01:02.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-14T18:01:02.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-14T18:01:02.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:01 smithi179 ceph-mon[33633]: osdmap e711: 8 total, 8 up, 8 in 2024-02-14T18:01:02.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:01:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:01:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:02 smithi179 ceph-mon[33633]: pgmap v1103: 300 pgs: 300 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:03.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:02 smithi100 ceph-mon[30138]: pgmap v1103: 300 pgs: 300 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:03.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:02 smithi100 ceph-mon[39557]: pgmap v1103: 300 pgs: 300 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:04.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:03 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:04.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:01:04.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:03 smithi179 ceph-mon[33633]: osdmap e712: 8 total, 8 up, 8 in 2024-02-14T18:01:04.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142"}]: dispatch 2024-02-14T18:01:04.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:03 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:04.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:01:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:03 smithi100 ceph-mon[30138]: osdmap e712: 8 total, 8 up, 8 in 2024-02-14T18:01:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142"}]: dispatch 2024-02-14T18:01:04.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:03 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:04.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:01:04.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:03 smithi100 ceph-mon[39557]: osdmap e712: 8 total, 8 up, 8 in 2024-02-14T18:01:04.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142"}]: dispatch 2024-02-14T18:01:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:04 smithi179 ceph-mon[33633]: pgmap v1106: 300 pgs: 300 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142"}]': finished 2024-02-14T18:01:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:04 smithi179 ceph-mon[33633]: osdmap e713: 8 total, 8 up, 8 in 2024-02-14T18:01:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142"}]: dispatch 2024-02-14T18:01:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:04 smithi100 ceph-mon[30138]: pgmap v1106: 300 pgs: 300 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142"}]': finished 2024-02-14T18:01:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:04 smithi100 ceph-mon[30138]: osdmap e713: 8 total, 8 up, 8 in 2024-02-14T18:01:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:05.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142"}]: dispatch 2024-02-14T18:01:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:04 smithi100 ceph-mon[39557]: pgmap v1106: 300 pgs: 300 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142"}]': finished 2024-02-14T18:01:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:04 smithi100 ceph-mon[39557]: osdmap e713: 8 total, 8 up, 8 in 2024-02-14T18:01:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-142"}]: dispatch 2024-02-14T18:01:06.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:05 smithi179 ceph-mon[33633]: osdmap e714: 8 total, 8 up, 8 in 2024-02-14T18:01:06.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:05 smithi100 ceph-mon[30138]: osdmap e714: 8 total, 8 up, 8 in 2024-02-14T18:01:06.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:05 smithi100 ceph-mon[39557]: osdmap e714: 8 total, 8 up, 8 in 2024-02-14T18:01:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:06 smithi179 ceph-mon[33633]: pgmap v1109: 268 pgs: 268 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:06 smithi179 ceph-mon[33633]: osdmap e715: 8 total, 8 up, 8 in 2024-02-14T18:01:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:07.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:06 smithi100 ceph-mon[30138]: pgmap v1109: 268 pgs: 268 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:07.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:06 smithi100 ceph-mon[30138]: osdmap e715: 8 total, 8 up, 8 in 2024-02-14T18:01:07.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:07.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:06 smithi100 ceph-mon[39557]: pgmap v1109: 268 pgs: 268 active+clean; 4.4 MiB data, 799 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:07.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:06 smithi100 ceph-mon[39557]: osdmap e715: 8 total, 8 up, 8 in 2024-02-14T18:01:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:07 smithi179 ceph-mon[33633]: osdmap e716: 8 total, 8 up, 8 in 2024-02-14T18:01:08.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:01:08.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:08.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:07 smithi100 ceph-mon[39557]: osdmap e716: 8 total, 8 up, 8 in 2024-02-14T18:01:08.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:01:08.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:08.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:07 smithi100 ceph-mon[30138]: osdmap e716: 8 total, 8 up, 8 in 2024-02-14T18:01:08.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:01:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:09 smithi179 ceph-mon[33633]: pgmap v1112: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T18:01:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:01:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:09 smithi179 ceph-mon[33633]: osdmap e717: 8 total, 8 up, 8 in 2024-02-14T18:01:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:09 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-144"}]: dispatch 2024-02-14T18:01:09.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:09 smithi100 ceph-mon[30138]: pgmap v1112: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T18:01:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:01:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:09 smithi100 ceph-mon[30138]: osdmap e717: 8 total, 8 up, 8 in 2024-02-14T18:01:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:09 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-144"}]: dispatch 2024-02-14T18:01:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:09 smithi100 ceph-mon[39557]: pgmap v1112: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T18:01:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:01:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:09 smithi100 ceph-mon[39557]: osdmap e717: 8 total, 8 up, 8 in 2024-02-14T18:01:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:09 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-144"}]: dispatch 2024-02-14T18:01:10.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-144"}]': finished 2024-02-14T18:01:10.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-mon[39557]: osdmap e718: 8 total, 8 up, 8 in 2024-02-14T18:01:10.300 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-144", "mode": "readproxy"}]: dispatch 2024-02-14T18:01:10.301 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:01:10.301 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-144", "mode": "readproxy"}]': finished 2024-02-14T18:01:10.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-144"}]': finished 2024-02-14T18:01:10.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-mon[30138]: osdmap e718: 8 total, 8 up, 8 in 2024-02-14T18:01:10.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-144", "mode": "readproxy"}]: dispatch 2024-02-14T18:01:10.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:01:10.301 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-144", "mode": "readproxy"}]': finished 2024-02-14T18:01:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-144"}]': finished 2024-02-14T18:01:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:10 smithi179 ceph-mon[33633]: osdmap e718: 8 total, 8 up, 8 in 2024-02-14T18:01:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-144", "mode": "readproxy"}]: dispatch 2024-02-14T18:01:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:10 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:01:10.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:10 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-144", "mode": "readproxy"}]': finished 2024-02-14T18:01:10.649 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:01:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:01:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:11 smithi179 ceph-mon[33633]: pgmap v1115: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T18:01:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:11 smithi179 ceph-mon[33633]: osdmap e719: 8 total, 8 up, 8 in 2024-02-14T18:01:11.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:11 smithi100 ceph-mon[30138]: pgmap v1115: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T18:01:11.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:11 smithi100 ceph-mon[30138]: osdmap e719: 8 total, 8 up, 8 in 2024-02-14T18:01:11.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:11 smithi100 ceph-mon[39557]: pgmap v1115: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-14T18:01:11.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:11 smithi100 ceph-mon[39557]: osdmap e719: 8 total, 8 up, 8 in 2024-02-14T18:01:12.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:01:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:01:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:13 smithi179 ceph-mon[33633]: pgmap v1117: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 B/s wr, 1 op/s 2024-02-14T18:01:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:13 smithi100 ceph-mon[39557]: pgmap v1117: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 B/s wr, 1 op/s 2024-02-14T18:01:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:13 smithi100 ceph-mon[30138]: pgmap v1117: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 211 B/s wr, 1 op/s 2024-02-14T18:01:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:14.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:14.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:15 smithi179 ceph-mon[33633]: pgmap v1118: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-02-14T18:01:15.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:15 smithi100 ceph-mon[39557]: pgmap v1118: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-02-14T18:01:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:15 smithi100 ceph-mon[30138]: pgmap v1118: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-02-14T18:01:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:17 smithi179 ceph-mon[33633]: pgmap v1119: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 784 B/s rd, 130 B/s wr, 0 op/s 2024-02-14T18:01:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:17 smithi100 ceph-mon[39557]: pgmap v1119: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 784 B/s rd, 130 B/s wr, 0 op/s 2024-02-14T18:01:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:17 smithi100 ceph-mon[30138]: pgmap v1119: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 784 B/s rd, 130 B/s wr, 0 op/s 2024-02-14T18:01:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:19 smithi179 ceph-mon[33633]: pgmap v1120: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 B/s wr, 1 op/s 2024-02-14T18:01:19.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:19 smithi100 ceph-mon[39557]: pgmap v1120: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 B/s wr, 1 op/s 2024-02-14T18:01:19.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:19 smithi100 ceph-mon[30138]: pgmap v1120: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 B/s wr, 1 op/s 2024-02-14T18:01:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:01:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:01:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:21 smithi179 ceph-mon[33633]: pgmap v1121: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-14T18:01:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:21.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:21 smithi100 ceph-mon[39557]: pgmap v1121: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-14T18:01:21.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:21 smithi100 ceph-mon[30138]: pgmap v1121: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-14T18:01:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:01:22.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:22 smithi179 ceph-mon[33633]: osdmap e720: 8 total, 8 up, 8 in 2024-02-14T18:01:22.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144"}]: dispatch 2024-02-14T18:01:22.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:22 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:01:22.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:22 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144"}]': finished 2024-02-14T18:01:22.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:22 smithi179 ceph-mon[33633]: osdmap e721: 8 total, 8 up, 8 in 2024-02-14T18:01:22.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:01:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[30138]: osdmap e720: 8 total, 8 up, 8 in 2024-02-14T18:01:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144"}]: dispatch 2024-02-14T18:01:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:01:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144"}]': finished 2024-02-14T18:01:22.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[30138]: osdmap e721: 8 total, 8 up, 8 in 2024-02-14T18:01:22.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:01:22.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[39557]: osdmap e720: 8 total, 8 up, 8 in 2024-02-14T18:01:22.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144"}]: dispatch 2024-02-14T18:01:22.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:01:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144"}]': finished 2024-02-14T18:01:22.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:22 smithi100 ceph-mon[39557]: osdmap e721: 8 total, 8 up, 8 in 2024-02-14T18:01:22.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:01:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:01:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:23 smithi179 ceph-mon[33633]: pgmap v1123: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:23 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144"}]: dispatch 2024-02-14T18:01:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:23 smithi100 ceph-mon[30138]: pgmap v1123: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:23 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144"}]: dispatch 2024-02-14T18:01:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:23 smithi100 ceph-mon[39557]: pgmap v1123: 300 pgs: 300 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:01:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:23 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-144"}]: dispatch 2024-02-14T18:01:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:24 smithi179 ceph-mon[33633]: osdmap e722: 8 total, 8 up, 8 in 2024-02-14T18:01:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:24 smithi100 ceph-mon[39557]: osdmap e722: 8 total, 8 up, 8 in 2024-02-14T18:01:24.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:24.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:24 smithi100 ceph-mon[30138]: osdmap e722: 8 total, 8 up, 8 in 2024-02-14T18:01:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:25 smithi100 ceph-mon[39557]: pgmap v1126: 268 pgs: 268 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:25 smithi100 ceph-mon[39557]: osdmap e723: 8 total, 8 up, 8 in 2024-02-14T18:01:25.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:25.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:25 smithi100 ceph-mon[30138]: pgmap v1126: 268 pgs: 268 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:25.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:25 smithi100 ceph-mon[30138]: osdmap e723: 8 total, 8 up, 8 in 2024-02-14T18:01:25.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:25 smithi179 ceph-mon[33633]: pgmap v1126: 268 pgs: 268 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:25 smithi179 ceph-mon[33633]: osdmap e723: 8 total, 8 up, 8 in 2024-02-14T18:01:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:26.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:26.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[39557]: osdmap e724: 8 total, 8 up, 8 in 2024-02-14T18:01:26.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:01:26.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:01:26.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[39557]: osdmap e725: 8 total, 8 up, 8 in 2024-02-14T18:01:26.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-146"}]: dispatch 2024-02-14T18:01:26.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:26.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[30138]: osdmap e724: 8 total, 8 up, 8 in 2024-02-14T18:01:26.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:01:26.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:01:26.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[30138]: osdmap e725: 8 total, 8 up, 8 in 2024-02-14T18:01:26.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-146"}]: dispatch 2024-02-14T18:01:26.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:26.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:26 smithi179 ceph-mon[33633]: osdmap e724: 8 total, 8 up, 8 in 2024-02-14T18:01:26.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:01:26.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:01:26.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:26 smithi179 ceph-mon[33633]: osdmap e725: 8 total, 8 up, 8 in 2024-02-14T18:01:26.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-146"}]: dispatch 2024-02-14T18:01:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:27 smithi100 ceph-mon[30138]: pgmap v1129: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-146"}]': finished 2024-02-14T18:01:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:27 smithi100 ceph-mon[30138]: osdmap e726: 8 total, 8 up, 8 in 2024-02-14T18:01:27.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-146", "mode": "writeback"}]: dispatch 2024-02-14T18:01:27.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:27 smithi100 ceph-mon[39557]: pgmap v1129: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:27.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-146"}]': finished 2024-02-14T18:01:27.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:27 smithi100 ceph-mon[39557]: osdmap e726: 8 total, 8 up, 8 in 2024-02-14T18:01:27.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-146", "mode": "writeback"}]: dispatch 2024-02-14T18:01:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:27 smithi179 ceph-mon[33633]: pgmap v1129: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 812 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:27.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi100-79838-111", "overlaypool": "test-rados-api-smithi100-79838-146"}]': finished 2024-02-14T18:01:27.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:27 smithi179 ceph-mon[33633]: osdmap e726: 8 total, 8 up, 8 in 2024-02-14T18:01:27.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-146", "mode": "writeback"}]: dispatch 2024-02-14T18:01:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:28 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:28 smithi179 ceph-mon[33633]: pgmap v1132: 300 pgs: 300 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T18:01:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:28 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:01:28.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-146", "mode": "writeback"}]': finished 2024-02-14T18:01:28.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:28 smithi179 ceph-mon[33633]: osdmap e727: 8 total, 8 up, 8 in 2024-02-14T18:01:28.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T18:01:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[39557]: pgmap v1132: 300 pgs: 300 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-146", "mode": "writeback"}]': finished 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[39557]: osdmap e727: 8 total, 8 up, 8 in 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[30138]: pgmap v1132: 300 pgs: 300 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi100-79838-146", "mode": "writeback"}]': finished 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[30138]: osdmap e727: 8 total, 8 up, 8 in 2024-02-14T18:01:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-14T18:01:30.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_count","val": "2"}]': finished 2024-02-14T18:01:30.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:30 smithi100 ceph-mon[30138]: osdmap e728: 8 total, 8 up, 8 in 2024-02-14T18:01:30.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T18:01:30.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_count","val": "2"}]': finished 2024-02-14T18:01:30.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:30 smithi100 ceph-mon[39557]: osdmap e728: 8 total, 8 up, 8 in 2024-02-14T18:01:30.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T18:01:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_count","val": "2"}]': finished 2024-02-14T18:01:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:30 smithi179 ceph-mon[33633]: osdmap e728: 8 total, 8 up, 8 in 2024-02-14T18:01:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-14T18:01:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:01:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:01:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:31.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:31 smithi100 ceph-mon[39557]: pgmap v1135: 300 pgs: 300 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T18:01:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_period","val": "600"}]': finished 2024-02-14T18:01:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:31 smithi100 ceph-mon[39557]: osdmap e729: 8 total, 8 up, 8 in 2024-02-14T18:01:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:31 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T18:01:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:31 smithi100 ceph-mon[30138]: pgmap v1135: 300 pgs: 300 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T18:01:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_period","val": "600"}]': finished 2024-02-14T18:01:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:31 smithi100 ceph-mon[30138]: osdmap e729: 8 total, 8 up, 8 in 2024-02-14T18:01:31.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:31 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T18:01:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:31 smithi179 ceph-mon[33633]: pgmap v1135: 300 pgs: 300 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-14T18:01:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_period","val": "600"}]': finished 2024-02-14T18:01:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:31 smithi179 ceph-mon[33633]: osdmap e729: 8 total, 8 up, 8 in 2024-02-14T18:01:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:31 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-14T18:01:32.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:01:32.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T18:01:32.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[39557]: osdmap e730: 8 total, 8 up, 8 in 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[39557]: osdmap e731: 8 total, 8 up, 8 in 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[30138]: osdmap e730: 8 total, 8 up, 8 in 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:01:32.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T18:01:32.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[30138]: osdmap e731: 8 total, 8 up, 8 in 2024-02-14T18:01:32.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:32 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T18:01:32.424 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:32 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-14T18:01:32.424 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-14T18:01:32.424 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:32 smithi179 ceph-mon[33633]: osdmap e730: 8 total, 8 up, 8 in 2024-02-14T18:01:32.424 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-14T18:01:32.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:32 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:01:32.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-14T18:01:32.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:32 smithi179 ceph-mon[33633]: osdmap e731: 8 total, 8 up, 8 in 2024-02-14T18:01:32.425 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:32 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-14T18:01:32.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:01:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:01:33.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[39557]: pgmap v1138: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T18:01:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:01:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:01:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "target_max_objects","val": "1"}]': finished 2024-02-14T18:01:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[39557]: osdmap e732: 8 total, 8 up, 8 in 2024-02-14T18:01:33.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[30138]: pgmap v1138: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T18:01:33.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:33.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:33.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:01:33.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:01:33.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:33.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "target_max_objects","val": "1"}]': finished 2024-02-14T18:01:33.401 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:33 smithi100 ceph-mon[30138]: osdmap e732: 8 total, 8 up, 8 in 2024-02-14T18:01:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:33 smithi179 ceph-mon[33633]: pgmap v1138: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T18:01:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:33 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:33 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:33 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:01:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:33 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:01:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:33 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:33 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-146","var": "target_max_objects","val": "1"}]': finished 2024-02-14T18:01:33.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:33 smithi179 ceph-mon[33633]: osdmap e732: 8 total, 8 up, 8 in 2024-02-14T18:01:34.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:34.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:35.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:35 smithi100 ceph-mon[39557]: pgmap v1141: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T18:01:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:35 smithi100 ceph-mon[39557]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T18:01:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:35 smithi100 ceph-mon[30138]: pgmap v1141: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T18:01:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:35 smithi100 ceph-mon[30138]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T18:01:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:35 smithi179 ceph-mon[33633]: pgmap v1141: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T18:01:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:35 smithi179 ceph-mon[33633]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-14T18:01:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:37 smithi100 ceph-mon[30138]: pgmap v1142: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T18:01:37.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:37 smithi100 ceph-mon[39557]: pgmap v1142: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T18:01:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:37 smithi179 ceph-mon[33633]: pgmap v1142: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-14T18:01:39.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:39 smithi100 ceph-mon[30138]: pgmap v1143: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T18:01:39.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:39 smithi100 ceph-mon[39557]: pgmap v1143: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T18:01:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:39 smithi179 ceph-mon[33633]: pgmap v1143: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-14T18:01:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:01:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:01:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:41.124 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:01:40 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=sqlstore.transactions t=2024-02-14T18:01:40.840728133Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-14T18:01:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:41 smithi100 ceph-mon[30138]: pgmap v1144: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-14T18:01:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:41 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]: dispatch 2024-02-14T18:01:41.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]: dispatch 2024-02-14T18:01:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:41 smithi100 ceph-mon[39557]: pgmap v1144: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-14T18:01:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:41 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]: dispatch 2024-02-14T18:01:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]: dispatch 2024-02-14T18:01:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:41 smithi179 ceph-mon[33633]: pgmap v1144: 300 pgs: 300 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-14T18:01:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:41 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]: dispatch 2024-02-14T18:01:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]: dispatch 2024-02-14T18:01:42.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:42 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]': finished 2024-02-14T18:01:42.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:42 smithi100 ceph-mon[30138]: osdmap e733: 8 total, 8 up, 8 in 2024-02-14T18:01:42.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:42 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]': finished 2024-02-14T18:01:42.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:42 smithi100 ceph-mon[39557]: osdmap e733: 8 total, 8 up, 8 in 2024-02-14T18:01:42.426 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:42 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "339.f", "id": [2, 6]}]': finished 2024-02-14T18:01:42.426 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:42 smithi179 ceph-mon[33633]: osdmap e733: 8 total, 8 up, 8 in 2024-02-14T18:01:42.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:01:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:01:43.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:43 smithi100 ceph-mon[30138]: pgmap v1146: 300 pgs: 1 peering, 299 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T18:01:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:43 smithi100 ceph-mon[30138]: osdmap e734: 8 total, 8 up, 8 in 2024-02-14T18:01:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:43 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:43 smithi100 ceph-mon[39557]: pgmap v1146: 300 pgs: 1 peering, 299 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T18:01:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:43 smithi100 ceph-mon[39557]: osdmap e734: 8 total, 8 up, 8 in 2024-02-14T18:01:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:43 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:43.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:43.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:43 smithi179 ceph-mon[33633]: pgmap v1146: 300 pgs: 1 peering, 299 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-14T18:01:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:43 smithi179 ceph-mon[33633]: osdmap e734: 8 total, 8 up, 8 in 2024-02-14T18:01:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:43 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:01:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:44 smithi100 ceph-mon[30138]: osdmap e735: 8 total, 8 up, 8 in 2024-02-14T18:01:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:44 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146"}]: dispatch 2024-02-14T18:01:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:01:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:44 smithi100 ceph-mon[39557]: osdmap e735: 8 total, 8 up, 8 in 2024-02-14T18:01:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:44 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146"}]: dispatch 2024-02-14T18:01:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:01:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:44 smithi179 ceph-mon[33633]: osdmap e735: 8 total, 8 up, 8 in 2024-02-14T18:01:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:44 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146"}]: dispatch 2024-02-14T18:01:45.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:45 smithi100 ceph-mon[39557]: pgmap v1149: 300 pgs: 1 peering, 299 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:45.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146"}]': finished 2024-02-14T18:01:45.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:45 smithi100 ceph-mon[39557]: osdmap e736: 8 total, 8 up, 8 in 2024-02-14T18:01:45.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:45.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146"}]: dispatch 2024-02-14T18:01:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:45 smithi100 ceph-mon[30138]: pgmap v1149: 300 pgs: 1 peering, 299 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146"}]': finished 2024-02-14T18:01:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:45 smithi100 ceph-mon[30138]: osdmap e736: 8 total, 8 up, 8 in 2024-02-14T18:01:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146"}]: dispatch 2024-02-14T18:01:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:45 smithi179 ceph-mon[33633]: pgmap v1149: 300 pgs: 1 peering, 299 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:01:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146"}]': finished 2024-02-14T18:01:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:45 smithi179 ceph-mon[33633]: osdmap e736: 8 total, 8 up, 8 in 2024-02-14T18:01:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-146"}]: dispatch 2024-02-14T18:01:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:46 smithi179 ceph-mon[33633]: osdmap e737: 8 total, 8 up, 8 in 2024-02-14T18:01:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:46 smithi179 ceph-mon[33633]: pgmap v1152: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:46.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:46 smithi100 ceph-mon[30138]: osdmap e737: 8 total, 8 up, 8 in 2024-02-14T18:01:46.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:46 smithi100 ceph-mon[30138]: pgmap v1152: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:46.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:46 smithi100 ceph-mon[39557]: osdmap e737: 8 total, 8 up, 8 in 2024-02-14T18:01:46.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:46 smithi100 ceph-mon[39557]: pgmap v1152: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:47 smithi179 ceph-mon[33633]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T18:01:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:47 smithi179 ceph-mon[33633]: osdmap e738: 8 total, 8 up, 8 in 2024-02-14T18:01:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:47 smithi100 ceph-mon[39557]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T18:01:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:47 smithi100 ceph-mon[39557]: osdmap e738: 8 total, 8 up, 8 in 2024-02-14T18:01:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:47 smithi100 ceph-mon[30138]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-14T18:01:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:47 smithi100 ceph-mon[30138]: osdmap e738: 8 total, 8 up, 8 in 2024-02-14T18:01:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:48 smithi179 ceph-mon[33633]: osdmap e739: 8 total, 8 up, 8 in 2024-02-14T18:01:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:01:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:48 smithi179 ceph-mon[33633]: pgmap v1155: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 850 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T18:01:48.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:48.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:48 smithi100 ceph-mon[30138]: osdmap e739: 8 total, 8 up, 8 in 2024-02-14T18:01:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:01:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:48 smithi100 ceph-mon[30138]: pgmap v1155: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 850 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T18:01:48.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:48.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:48 smithi100 ceph-mon[39557]: osdmap e739: 8 total, 8 up, 8 in 2024-02-14T18:01:48.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-14T18:01:48.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:48 smithi100 ceph-mon[39557]: pgmap v1155: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 850 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-14T18:01:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:01:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:49 smithi179 ceph-mon[33633]: osdmap e740: 8 total, 8 up, 8 in 2024-02-14T18:01:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148"}]: dispatch 2024-02-14T18:01:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:01:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:49 smithi100 ceph-mon[30138]: osdmap e740: 8 total, 8 up, 8 in 2024-02-14T18:01:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148"}]: dispatch 2024-02-14T18:01:49.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-14T18:01:49.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:49 smithi100 ceph-mon[39557]: osdmap e740: 8 total, 8 up, 8 in 2024-02-14T18:01:49.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148"}]: dispatch 2024-02-14T18:01:50.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148"}]': finished 2024-02-14T18:01:50.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:50 smithi179 ceph-mon[33633]: osdmap e741: 8 total, 8 up, 8 in 2024-02-14T18:01:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:50 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148"}]: dispatch 2024-02-14T18:01:50.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:50 smithi179 ceph-mon[33633]: pgmap v1158: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 850 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:50.649 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:01:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:01:50.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148"}]': finished 2024-02-14T18:01:50.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-mon[39557]: osdmap e741: 8 total, 8 up, 8 in 2024-02-14T18:01:50.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:50.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148"}]: dispatch 2024-02-14T18:01:50.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-mon[39557]: pgmap v1158: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 850 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:50.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148"}]': finished 2024-02-14T18:01:50.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-mon[30138]: osdmap e741: 8 total, 8 up, 8 in 2024-02-14T18:01:50.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:50.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-148"}]: dispatch 2024-02-14T18:01:50.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:50 smithi100 ceph-mon[30138]: pgmap v1158: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 850 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:01:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:51 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:01:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:51 smithi179 ceph-mon[33633]: osdmap e742: 8 total, 8 up, 8 in 2024-02-14T18:01:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:51 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:01:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:51 smithi100 ceph-mon[30138]: osdmap e742: 8 total, 8 up, 8 in 2024-02-14T18:01:51.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:51 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:01:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:51 smithi100 ceph-mon[39557]: osdmap e742: 8 total, 8 up, 8 in 2024-02-14T18:01:52.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:01:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:01:52.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:52 smithi179 ceph-mon[33633]: osdmap e743: 8 total, 8 up, 8 in 2024-02-14T18:01:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:52 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:52.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:52 smithi179 ceph-mon[33633]: pgmap v1161: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 868 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T18:01:52.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:52 smithi100 ceph-mon[30138]: osdmap e743: 8 total, 8 up, 8 in 2024-02-14T18:01:52.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:52 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:52.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:52 smithi100 ceph-mon[30138]: pgmap v1161: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 868 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T18:01:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:52 smithi100 ceph-mon[39557]: osdmap e743: 8 total, 8 up, 8 in 2024-02-14T18:01:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:52 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:52 smithi100 ceph-mon[39557]: pgmap v1161: 300 pgs: 32 creating+peering, 268 active+clean; 4.4 MiB data, 868 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T18:01:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:53 smithi179 ceph-mon[33633]: osdmap e744: 8 total, 8 up, 8 in 2024-02-14T18:01:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:53 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-150"}]: dispatch 2024-02-14T18:01:53.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:53 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:53 smithi100 ceph-mon[30138]: osdmap e744: 8 total, 8 up, 8 in 2024-02-14T18:01:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:53 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-150"}]: dispatch 2024-02-14T18:01:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:53 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:53.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:53 smithi100 ceph-mon[39557]: osdmap e744: 8 total, 8 up, 8 in 2024-02-14T18:01:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:53 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-150"}]: dispatch 2024-02-14T18:01:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:53 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:01:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:54 smithi179 ceph-mon[33633]: osdmap e745: 8 total, 8 up, 8 in 2024-02-14T18:01:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:54 smithi179 ceph-mon[33633]: pgmap v1164: 268 pgs: 268 active+clean; 4.4 MiB data, 868 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T18:01:54.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:54 smithi100 ceph-mon[30138]: osdmap e745: 8 total, 8 up, 8 in 2024-02-14T18:01:54.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:54 smithi100 ceph-mon[30138]: pgmap v1164: 268 pgs: 268 active+clean; 4.4 MiB data, 868 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T18:01:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:54 smithi100 ceph-mon[39557]: osdmap e745: 8 total, 8 up, 8 in 2024-02-14T18:01:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:54 smithi100 ceph-mon[39557]: pgmap v1164: 268 pgs: 268 active+clean; 4.4 MiB data, 868 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T18:01:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:55 smithi179 ceph-mon[33633]: osdmap e746: 8 total, 8 up, 8 in 2024-02-14T18:01:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:55 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:55 smithi100 ceph-mon[30138]: osdmap e746: 8 total, 8 up, 8 in 2024-02-14T18:01:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:55 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:55 smithi100 ceph-mon[39557]: osdmap e746: 8 total, 8 up, 8 in 2024-02-14T18:01:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:55 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:56 smithi179 ceph-mon[33633]: osdmap e747: 8 total, 8 up, 8 in 2024-02-14T18:01:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:56 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-152"}]: dispatch 2024-02-14T18:01:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:56 smithi179 ceph-mon[33633]: pgmap v1167: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 868 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:56.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:56.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:56 smithi100 ceph-mon[30138]: osdmap e747: 8 total, 8 up, 8 in 2024-02-14T18:01:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:56 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-152"}]: dispatch 2024-02-14T18:01:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:56 smithi100 ceph-mon[30138]: pgmap v1167: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 868 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:56 smithi100 ceph-mon[39557]: osdmap e747: 8 total, 8 up, 8 in 2024-02-14T18:01:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:56 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-152"}]: dispatch 2024-02-14T18:01:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:56 smithi100 ceph-mon[39557]: pgmap v1167: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 868 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:01:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:57 smithi179 ceph-mon[33633]: osdmap e748: 8 total, 8 up, 8 in 2024-02-14T18:01:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:57 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:57 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:57.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:57 smithi100 ceph-mon[30138]: osdmap e748: 8 total, 8 up, 8 in 2024-02-14T18:01:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:57 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:57 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:57 smithi100 ceph-mon[39557]: osdmap e748: 8 total, 8 up, 8 in 2024-02-14T18:01:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:57 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:01:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:57 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:01:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:58 smithi179 ceph-mon[33633]: osdmap e749: 8 total, 8 up, 8 in 2024-02-14T18:01:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:58 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:58 smithi179 ceph-mon[33633]: pgmap v1170: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 869 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-14T18:01:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:58 smithi100 ceph-mon[30138]: osdmap e749: 8 total, 8 up, 8 in 2024-02-14T18:01:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:58 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:58 smithi100 ceph-mon[30138]: pgmap v1170: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 869 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-14T18:01:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:58 smithi100 ceph-mon[39557]: osdmap e749: 8 total, 8 up, 8 in 2024-02-14T18:01:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:58 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:01:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:58 smithi100 ceph-mon[39557]: pgmap v1170: 300 pgs: 32 unknown, 268 active+clean; 4.4 MiB data, 869 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-14T18:01:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:59 smithi179 ceph-mon[33633]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:01:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:59 smithi179 ceph-mon[33633]: osdmap e750: 8 total, 8 up, 8 in 2024-02-14T18:01:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-111","var": "dedup_tier","val": "test-rados-api-smithi100-79838-154"}]: dispatch 2024-02-14T18:01:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:59.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:01:59 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-154"}]: dispatch 2024-02-14T18:01:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[30138]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:01:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[30138]: osdmap e750: 8 total, 8 up, 8 in 2024-02-14T18:01:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-111","var": "dedup_tier","val": "test-rados-api-smithi100-79838-154"}]: dispatch 2024-02-14T18:01:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-154"}]: dispatch 2024-02-14T18:01:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[39557]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:01:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi100-79838-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:01:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[39557]: osdmap e750: 8 total, 8 up, 8 in 2024-02-14T18:01:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi100-79838-111","var": "dedup_tier","val": "test-rados-api-smithi100-79838-154"}]: dispatch 2024-02-14T18:01:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:01:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:01:59 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi100-79838-111", "tierpool": "test-rados-api-smithi100-79838-154"}]: dispatch 2024-02-14T18:02:00.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:00 smithi100 ceph-mon[30138]: osdmap e751: 8 total, 8 up, 8 in 2024-02-14T18:02:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:00 smithi100 ceph-mon[30138]: pgmap v1173: 268 pgs: 268 active+clean; 4.4 MiB data, 869 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-14T18:02:00.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:02:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:02:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:02:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:00 smithi100 ceph-mon[39557]: osdmap e751: 8 total, 8 up, 8 in 2024-02-14T18:02:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:00 smithi100 ceph-mon[39557]: pgmap v1173: 268 pgs: 268 active+clean; 4.4 MiB data, 869 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-14T18:02:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:00 smithi179 ceph-mon[33633]: osdmap e751: 8 total, 8 up, 8 in 2024-02-14T18:02:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:00 smithi179 ceph-mon[33633]: pgmap v1173: 268 pgs: 268 active+clean; 4.4 MiB data, 869 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-14T18:02:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:01 smithi100 ceph-mon[30138]: osdmap e752: 8 total, 8 up, 8 in 2024-02-14T18:02:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:02:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:01 smithi100 ceph-mon[39557]: osdmap e752: 8 total, 8 up, 8 in 2024-02-14T18:02:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:02:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:01 smithi179 ceph-mon[33633]: osdmap e752: 8 total, 8 up, 8 in 2024-02-14T18:02:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:02:02.347 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting LibRadosTwoPoolsECPP.TryFlush (7818 ms) 2024-02-14T18:02:02.347 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-14T18:02:02.347 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12602 ms) 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8048 ms) 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13074 ms) 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7318 ms) 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8058 ms) 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: ok, hit_set contains 335:602f83fe:::foo:head 2024-02-14T18:02:02.348 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (8107 ms) 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: first is 1707933638 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,1707933640,1707933641,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,1707933640,1707933641,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,1707933640,1707933641,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,1707933640,1707933641,1707933643,1707933644,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,1707933640,1707933641,1707933643,1707933644,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,1707933640,1707933641,1707933643,1707933644,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,1707933640,1707933641,1707933643,1707933644,1707933646,1707933647,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,1707933640,1707933641,1707933643,1707933644,1707933646,1707933647,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933638,1707933640,1707933641,1707933643,1707933644,1707933646,1707933647,0 2024-02-14T18:02:02.349 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: got ls 1707933641,1707933643,1707933644,1707933646,1707933647,1707933649,1707933650,0 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: first now 1707933641, trimmed 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20224 ms) 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: foo0 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14073 ms) 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18079 ms) 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22163 ms) 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5046 ms) 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3012 ms) 2024-02-14T18:02:02.350 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-14T18:02:02.351 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3031 ms) 2024-02-14T18:02:02.351 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-14T18:02:02.351 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3012 ms) 2024-02-14T18:02:02.351 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (228972 ms total) 2024-02-14T18:02:02.351 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: 2024-02-14T18:02:02.351 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-14T18:02:02.351 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (856253 ms total) 2024-02-14T18:02:02.351 INFO:tasks.workunit.client.0.smithi100.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-02-14T18:02:02.352 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:02:02.352 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79570 2024-02-14T18:02:02.352 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79570 2024-02-14T18:02:02.352 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:02:02.352 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79742 2024-02-14T18:02:02.352 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79742 2024-02-14T18:02:02.352 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:02:02.352 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80308 2024-02-14T18:02:02.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80308 2024-02-14T18:02:02.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:02:02.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80008 2024-02-14T18:02:02.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80008 2024-02-14T18:02:02.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:02:02.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79884 2024-02-14T18:02:02.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79884 2024-02-14T18:02:02.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:02:02.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79652 2024-02-14T18:02:02.353 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79652 2024-02-14T18:02:02.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:02:02.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:02 smithi100 ceph-mon[30138]: osdmap e753: 8 total, 8 up, 8 in 2024-02-14T18:02:02.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:02:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:02 smithi100 ceph-mon[30138]: pgmap v1176: 260 pgs: 260 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:02:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:02:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:02 smithi100 ceph-mon[39557]: osdmap e753: 8 total, 8 up, 8 in 2024-02-14T18:02:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:02:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:02 smithi100 ceph-mon[39557]: pgmap v1176: 260 pgs: 260 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:02:02.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:02:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:02:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:02:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:02 smithi179 ceph-mon[33633]: osdmap e753: 8 total, 8 up, 8 in 2024-02-14T18:02:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-111"}]: dispatch 2024-02-14T18:02:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:02 smithi179 ceph-mon[33633]: pgmap v1176: 260 pgs: 260 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:02:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:02:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:03 smithi100 ceph-mon[30138]: osdmap e754: 8 total, 8 up, 8 in 2024-02-14T18:02:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:03 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:02:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:03 smithi100 ceph-mon[39557]: osdmap e754: 8 total, 8 up, 8 in 2024-02-14T18:02:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:03 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1274895002' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi100-79838-111"}]': finished 2024-02-14T18:02:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:03 smithi179 ceph-mon[33633]: osdmap e754: 8 total, 8 up, 8 in 2024-02-14T18:02:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:03 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:04 smithi179 ceph-mon[33633]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:02:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:04 smithi179 ceph-mon[33633]: pgmap v1178: 260 pgs: 260 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:02:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:04 smithi100 ceph-mon[30138]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:02:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:04 smithi100 ceph-mon[30138]: pgmap v1178: 260 pgs: 260 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:02:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:04 smithi100 ceph-mon[39557]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:02:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:04 smithi100 ceph-mon[39557]: pgmap v1178: 260 pgs: 260 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:02:05.564 INFO:tasks.workunit.client.0.smithi100.stdout: 1 sec 2024-02-14T18:02:05.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.564 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.565 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.566 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.567 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.568 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.569 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.570 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.571 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.571 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.571 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:05.571 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:02:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:06 smithi179 ceph-mon[33633]: pgmap v1179: 260 pgs: 260 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:02:07.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:06 smithi100 ceph-mon[30138]: pgmap v1179: 260 pgs: 260 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:02:07.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:06 smithi100 ceph-mon[39557]: pgmap v1179: 260 pgs: 260 active+clean; 582 KiB data, 865 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:02:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:08 smithi179 ceph-mon[33633]: pgmap v1180: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-14T18:02:09.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:08 smithi100 ceph-mon[30138]: pgmap v1180: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-14T18:02:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:08 smithi100 ceph-mon[39557]: pgmap v1180: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-14T18:02:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:02:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:02:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:02:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:10 smithi179 ceph-mon[33633]: pgmap v1181: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:11.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:10 smithi100 ceph-mon[30138]: pgmap v1181: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:11.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:10 smithi100 ceph-mon[39557]: pgmap v1181: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:12.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:02:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:02:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:12 smithi179 ceph-mon[33633]: pgmap v1182: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:02:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:12 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:12 smithi100 ceph-mon[30138]: pgmap v1182: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:02:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:12 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:12 smithi100 ceph-mon[39557]: pgmap v1182: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:02:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:12 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:13 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:13 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:14.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:13 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:14 smithi179 ceph-mon[33633]: pgmap v1183: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 889 B/s rd, 0 op/s 2024-02-14T18:02:15.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:14 smithi100 ceph-mon[30138]: pgmap v1183: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 889 B/s rd, 0 op/s 2024-02-14T18:02:15.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:14 smithi100 ceph-mon[39557]: pgmap v1183: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 889 B/s rd, 0 op/s 2024-02-14T18:02:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:16 smithi179 ceph-mon[33633]: pgmap v1184: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:17.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:16 smithi100 ceph-mon[30138]: pgmap v1184: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:16 smithi100 ceph-mon[39557]: pgmap v1184: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:18 smithi179 ceph-mon[33633]: pgmap v1185: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:19.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:18 smithi100 ceph-mon[30138]: pgmap v1185: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:19.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:18 smithi100 ceph-mon[39557]: pgmap v1185: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:02:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:02:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:02:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:20 smithi179 ceph-mon[33633]: pgmap v1186: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:21.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:20 smithi100 ceph-mon[30138]: pgmap v1186: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:21.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:20 smithi100 ceph-mon[39557]: pgmap v1186: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:22.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:02:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:02:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:22 smithi179 ceph-mon[33633]: pgmap v1187: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:22 smithi100 ceph-mon[30138]: pgmap v1187: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:22 smithi100 ceph-mon[39557]: pgmap v1187: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:23 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:24.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:23 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:23 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:24 smithi179 ceph-mon[33633]: pgmap v1188: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:24 smithi100 ceph-mon[30138]: pgmap v1188: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:24 smithi100 ceph-mon[39557]: pgmap v1188: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:26 smithi179 ceph-mon[33633]: pgmap v1189: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:26 smithi100 ceph-mon[30138]: pgmap v1189: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:27.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:26 smithi100 ceph-mon[39557]: pgmap v1189: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:28.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:27 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:28.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:27 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:28.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:27 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:29.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:28 smithi179 ceph-mon[33633]: pgmap v1190: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:29.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:28 smithi100 ceph-mon[30138]: pgmap v1190: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:29.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:28 smithi100 ceph-mon[39557]: pgmap v1190: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:02:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:02:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:02:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:30 smithi179 ceph-mon[33633]: pgmap v1191: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:31.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:30 smithi100 ceph-mon[30138]: pgmap v1191: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:30 smithi100 ceph-mon[39557]: pgmap v1191: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:32.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:02:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:02:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:32 smithi179 ceph-mon[33633]: pgmap v1192: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:33.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:32 smithi100 ceph-mon[39557]: pgmap v1192: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:33.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:32 smithi100 ceph-mon[30138]: pgmap v1192: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:33 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:33 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:02:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:33 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:02:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:33 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:02:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:33 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:02:34.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:33 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:34.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:33 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:02:34.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:33 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:02:34.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:33 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:02:34.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:33 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:02:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:33 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:33 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:02:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:33 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:02:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:33 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:02:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:33 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:02:35.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:34 smithi179 ceph-mon[33633]: pgmap v1193: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:34 smithi100 ceph-mon[39557]: pgmap v1193: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:35.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:34 smithi100 ceph-mon[30138]: pgmap v1193: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:36 smithi179 ceph-mon[33633]: pgmap v1194: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:37.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:36 smithi100 ceph-mon[39557]: pgmap v1194: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:37.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:36 smithi100 ceph-mon[30138]: pgmap v1194: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:38 smithi179 ceph-mon[33633]: pgmap v1195: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:39.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:38 smithi100 ceph-mon[30138]: pgmap v1195: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:39.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:38 smithi100 ceph-mon[39557]: pgmap v1195: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:02:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:02:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:02:41.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:40 smithi179 ceph-mon[33633]: pgmap v1196: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:40 smithi100 ceph-mon[30138]: pgmap v1196: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:40 smithi100 ceph-mon[39557]: pgmap v1196: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:42.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:02:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:02:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:42 smithi179 ceph-mon[33633]: pgmap v1197: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:42 smithi100 ceph-mon[39557]: pgmap v1197: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:42 smithi100 ceph-mon[30138]: pgmap v1197: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:44.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:43 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:44.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:43 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:43 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:44 smithi179 ceph-mon[33633]: pgmap v1198: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:45.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:44 smithi100 ceph-mon[30138]: pgmap v1198: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:45.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:44 smithi100 ceph-mon[39557]: pgmap v1198: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:47.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:46 smithi179 ceph-mon[33633]: pgmap v1199: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:47.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:46 smithi100 ceph-mon[39557]: pgmap v1199: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:47.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:46 smithi100 ceph-mon[30138]: pgmap v1199: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:48 smithi179 ceph-mon[33633]: pgmap v1200: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:48 smithi100 ceph-mon[30138]: pgmap v1200: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:49.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:48 smithi100 ceph-mon[39557]: pgmap v1200: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:02:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:02:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:02:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:50 smithi179 ceph-mon[33633]: pgmap v1201: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:50 smithi100 ceph-mon[30138]: pgmap v1201: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:51.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:50 smithi100 ceph-mon[39557]: pgmap v1201: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:52.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:02:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:02:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:52 smithi179 ceph-mon[33633]: pgmap v1202: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:52 smithi100 ceph-mon[30138]: pgmap v1202: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:53.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:52 smithi100 ceph-mon[39557]: pgmap v1202: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:54.323 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:53 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:54.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:02:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:55 smithi179 ceph-mon[33633]: pgmap v1203: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:55.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:55 smithi100 ceph-mon[30138]: pgmap v1203: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:55 smithi100 ceph-mon[39557]: pgmap v1203: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:02:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:57 smithi179 ceph-mon[33633]: pgmap v1204: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:57.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:57 smithi100 ceph-mon[30138]: pgmap v1204: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:57.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:57 smithi100 ceph-mon[39557]: pgmap v1204: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:02:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:58.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:02:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:02:59 smithi179 ceph-mon[33633]: pgmap v1205: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:59.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:02:59 smithi100 ceph-mon[39557]: pgmap v1205: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:02:59.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:02:59 smithi100 ceph-mon[30138]: pgmap v1205: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:03:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:03:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:03:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:01 smithi179 ceph-mon[33633]: pgmap v1206: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:01.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:01 smithi100 ceph-mon[39557]: pgmap v1206: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:01 smithi100 ceph-mon[30138]: pgmap v1206: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:02.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:03:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:03:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:03 smithi179 ceph-mon[33633]: pgmap v1207: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:03 smithi100 ceph-mon[39557]: pgmap v1207: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:03 smithi100 ceph-mon[30138]: pgmap v1207: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:04.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:04.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:04.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:05.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:05 smithi179 ceph-mon[33633]: pgmap v1208: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:05 smithi100 ceph-mon[30138]: pgmap v1208: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:05.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:05 smithi100 ceph-mon[39557]: pgmap v1208: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:07 smithi179 ceph-mon[33633]: pgmap v1209: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:07.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:07 smithi100 ceph-mon[30138]: pgmap v1209: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:07.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:07 smithi100 ceph-mon[39557]: pgmap v1209: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:09 smithi179 ceph-mon[33633]: pgmap v1210: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:09.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:09 smithi100 ceph-mon[30138]: pgmap v1210: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:09 smithi100 ceph-mon[39557]: pgmap v1210: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:03:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:03:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:03:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:11 smithi179 ceph-mon[33633]: pgmap v1211: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:11.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:11 smithi100 ceph-mon[30138]: pgmap v1211: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:11.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:11 smithi100 ceph-mon[39557]: pgmap v1211: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:12.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:03:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:03:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:13 smithi179 ceph-mon[33633]: pgmap v1212: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:13 smithi100 ceph-mon[30138]: pgmap v1212: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:13 smithi100 ceph-mon[39557]: pgmap v1212: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:14.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:14.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:15 smithi179 ceph-mon[33633]: pgmap v1213: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:15.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:15 smithi100 ceph-mon[39557]: pgmap v1213: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:15.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:15 smithi100 ceph-mon[30138]: pgmap v1213: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:17 smithi179 ceph-mon[33633]: pgmap v1214: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:17 smithi100 ceph-mon[39557]: pgmap v1214: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:17.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:17 smithi100 ceph-mon[30138]: pgmap v1214: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:19 smithi179 ceph-mon[33633]: pgmap v1215: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:19.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:19 smithi100 ceph-mon[30138]: pgmap v1215: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:19.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:19 smithi100 ceph-mon[39557]: pgmap v1215: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:03:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:03:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:03:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:21 smithi179 ceph-mon[33633]: pgmap v1216: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:21.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:21 smithi100 ceph-mon[39557]: pgmap v1216: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:21.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:21 smithi100 ceph-mon[30138]: pgmap v1216: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:22.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:03:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:03:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:23 smithi179 ceph-mon[33633]: pgmap v1217: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:23 smithi100 ceph-mon[39557]: pgmap v1217: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:23 smithi100 ceph-mon[30138]: pgmap v1217: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:24.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:25 smithi179 ceph-mon[33633]: pgmap v1218: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:25 smithi100 ceph-mon[30138]: pgmap v1218: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:25 smithi100 ceph-mon[39557]: pgmap v1218: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:27 smithi179 ceph-mon[33633]: pgmap v1219: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:27.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:27 smithi100 ceph-mon[39557]: pgmap v1219: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:27 smithi100 ceph-mon[30138]: pgmap v1219: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:28.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:28.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:29.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:29 smithi100 ceph-mon[30138]: pgmap v1220: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:29.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:29 smithi100 ceph-mon[39557]: pgmap v1220: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:29 smithi179 ceph-mon[33633]: pgmap v1220: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:03:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:03:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:03:31.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:31 smithi100 ceph-mon[30138]: pgmap v1221: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:31.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:31 smithi100 ceph-mon[39557]: pgmap v1221: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:31 smithi179 ceph-mon[33633]: pgmap v1221: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:32.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:03:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:03:33.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:33 smithi100 ceph-mon[30138]: pgmap v1222: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:33 smithi100 ceph-mon[39557]: pgmap v1222: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:33 smithi179 ceph-mon[33633]: pgmap v1222: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:34.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:34.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:34 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:03:34.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:34.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:34 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:03:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:34 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:03:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:35 smithi100 ceph-mon[30138]: pgmap v1223: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:35 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:03:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:35 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:03:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:35 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:03:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:35 smithi100 ceph-mon[39557]: pgmap v1223: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:35 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:03:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:35 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:03:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:35 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:03:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:35 smithi179 ceph-mon[33633]: pgmap v1223: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:35 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:03:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:35 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:03:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:35 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:03:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:37 smithi100 ceph-mon[30138]: pgmap v1224: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:37.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:37 smithi100 ceph-mon[39557]: pgmap v1224: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:37 smithi179 ceph-mon[33633]: pgmap v1224: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:39.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:39 smithi100 ceph-mon[30138]: pgmap v1225: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:39.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:39 smithi100 ceph-mon[39557]: pgmap v1225: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:39 smithi179 ceph-mon[33633]: pgmap v1225: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:03:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:03:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:03:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:41 smithi100 ceph-mon[30138]: pgmap v1226: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:41 smithi100 ceph-mon[39557]: pgmap v1226: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:41 smithi179 ceph-mon[33633]: pgmap v1226: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:42.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:03:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:03:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:43 smithi100 ceph-mon[39557]: pgmap v1227: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:43 smithi100 ceph-mon[30138]: pgmap v1227: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:43.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:43 smithi179 ceph-mon[33633]: pgmap v1227: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:44.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:44.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:45.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:45 smithi100 ceph-mon[39557]: pgmap v1228: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:45 smithi100 ceph-mon[30138]: pgmap v1228: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:45 smithi179 ceph-mon[33633]: pgmap v1228: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:45.576 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.576 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.576 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.576 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.576 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.576 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.576 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.576 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.576 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.577 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.578 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.579 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.580 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.581 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.582 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:45.583 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:03:47.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:47 smithi100 ceph-mon[39557]: pgmap v1229: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:47.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:47 smithi100 ceph-mon[30138]: pgmap v1229: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:47 smithi179 ceph-mon[33633]: pgmap v1229: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:49 smithi100 ceph-mon[30138]: pgmap v1230: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:49.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:49 smithi100 ceph-mon[39557]: pgmap v1230: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:49 smithi179 ceph-mon[33633]: pgmap v1230: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:03:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:03:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:03:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:51 smithi100 ceph-mon[30138]: pgmap v1231: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:51.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:51 smithi100 ceph-mon[39557]: pgmap v1231: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:51 smithi179 ceph-mon[33633]: pgmap v1231: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:52.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:03:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:03:53.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:53 smithi100 ceph-mon[39557]: pgmap v1232: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:53 smithi100 ceph-mon[30138]: pgmap v1232: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:53 smithi179 ceph-mon[33633]: pgmap v1232: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:54.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:03:55.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:55 smithi100 ceph-mon[30138]: pgmap v1233: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:55 smithi100 ceph-mon[39557]: pgmap v1233: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:55 smithi179 ceph-mon[33633]: pgmap v1233: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:03:57.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:57 smithi100 ceph-mon[30138]: pgmap v1234: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:57.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:57 smithi100 ceph-mon[39557]: pgmap v1234: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:57 smithi179 ceph-mon[33633]: pgmap v1234: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:03:58.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:03:59.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:03:59 smithi100 ceph-mon[30138]: pgmap v1235: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:59.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:03:59 smithi100 ceph-mon[39557]: pgmap v1235: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:03:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:03:59 smithi179 ceph-mon[33633]: pgmap v1235: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:00.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:04:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:04:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:04:01.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:01 smithi100 ceph-mon[30138]: pgmap v1236: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:01.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:01 smithi100 ceph-mon[39557]: pgmap v1236: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:01 smithi179 ceph-mon[33633]: pgmap v1236: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:02.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:04:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:04:03.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:03 smithi100 ceph-mon[30138]: pgmap v1237: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:03.401 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:03 smithi100 ceph-mon[39557]: pgmap v1237: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:03 smithi179 ceph-mon[33633]: pgmap v1237: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:05 smithi179 ceph-mon[33633]: pgmap v1238: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:05 smithi100 ceph-mon[30138]: pgmap v1238: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:05 smithi100 ceph-mon[39557]: pgmap v1238: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:06 smithi179 ceph-mon[33633]: pgmap v1239: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:06.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:06 smithi100 ceph-mon[30138]: pgmap v1239: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:06.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:06 smithi100 ceph-mon[39557]: pgmap v1239: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:08 smithi179 ceph-mon[33633]: pgmap v1240: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:09.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:08 smithi100 ceph-mon[30138]: pgmap v1240: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:08 smithi100 ceph-mon[39557]: pgmap v1240: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:04:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:04:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:04:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:10 smithi179 ceph-mon[33633]: pgmap v1241: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:11.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:10 smithi100 ceph-mon[30138]: pgmap v1241: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:11.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:10 smithi100 ceph-mon[39557]: pgmap v1241: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:12.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:04:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:04:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:12 smithi179 ceph-mon[33633]: pgmap v1242: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:12 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:12 smithi100 ceph-mon[30138]: pgmap v1242: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:12 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:12 smithi100 ceph-mon[39557]: pgmap v1242: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:12 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:13 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:14.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:13 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:14.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:13 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:14 smithi179 ceph-mon[33633]: pgmap v1243: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:15.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:14 smithi100 ceph-mon[30138]: pgmap v1243: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:15.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:14 smithi100 ceph-mon[39557]: pgmap v1243: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:16 smithi179 ceph-mon[33633]: pgmap v1244: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:17.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:16 smithi100 ceph-mon[30138]: pgmap v1244: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:16 smithi100 ceph-mon[39557]: pgmap v1244: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:18 smithi179 ceph-mon[33633]: pgmap v1245: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:19.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:18 smithi100 ceph-mon[30138]: pgmap v1245: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:19.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:18 smithi100 ceph-mon[39557]: pgmap v1245: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:20.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:04:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:04:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:04:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:20 smithi179 ceph-mon[33633]: pgmap v1246: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:21.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:20 smithi100 ceph-mon[30138]: pgmap v1246: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:20 smithi100 ceph-mon[39557]: pgmap v1246: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:22.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:04:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:04:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:22 smithi179 ceph-mon[33633]: pgmap v1247: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:22 smithi100 ceph-mon[39557]: pgmap v1247: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:22 smithi100 ceph-mon[30138]: pgmap v1247: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:23 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:24.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:23 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:23 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:24 smithi179 ceph-mon[33633]: pgmap v1248: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:24 smithi100 ceph-mon[30138]: pgmap v1248: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:24 smithi100 ceph-mon[39557]: pgmap v1248: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:26 smithi179 ceph-mon[33633]: pgmap v1249: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:26 smithi100 ceph-mon[30138]: pgmap v1249: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:27.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:26 smithi100 ceph-mon[39557]: pgmap v1249: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:28.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:27 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:28.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:27 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:28.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:27 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:29.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:28 smithi179 ceph-mon[33633]: pgmap v1250: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:29.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:28 smithi100 ceph-mon[39557]: pgmap v1250: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:29.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:28 smithi100 ceph-mon[30138]: pgmap v1250: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:04:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:04:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:04:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:30 smithi179 ceph-mon[33633]: pgmap v1251: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:31.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:31 smithi100 ceph-mon[39557]: pgmap v1251: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:31.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:30 smithi100 ceph-mon[30138]: pgmap v1251: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:32.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:04:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:04:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:33 smithi179 ceph-mon[33633]: pgmap v1252: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:33.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:33 smithi100 ceph-mon[30138]: pgmap v1252: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:33.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:33 smithi100 ceph-mon[39557]: pgmap v1252: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:34.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:34.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:35 smithi179 ceph-mon[33633]: pgmap v1253: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:35 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:04:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:35 smithi100 ceph-mon[30138]: pgmap v1253: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:35 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:04:35.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:35 smithi100 ceph-mon[39557]: pgmap v1253: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:35.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:35 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:04:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:36 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:04:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:36 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:04:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:36 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:04:36.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:36 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:04:36.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:36 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:04:36.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:36 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:04:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:36 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:04:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:36 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:04:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:36 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:04:37.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:37 smithi179 ceph-mon[33633]: pgmap v1254: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:37 smithi100 ceph-mon[30138]: pgmap v1254: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:37 smithi100 ceph-mon[39557]: pgmap v1254: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:39.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:39 smithi179 ceph-mon[33633]: pgmap v1255: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:39.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:39 smithi100 ceph-mon[30138]: pgmap v1255: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:39.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:39 smithi100 ceph-mon[39557]: pgmap v1255: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:04:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:04:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:04:41.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:41 smithi179 ceph-mon[33633]: pgmap v1256: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:41 smithi100 ceph-mon[30138]: pgmap v1256: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:41.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:41 smithi100 ceph-mon[39557]: pgmap v1256: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:43.024 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:04:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:04:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:43 smithi179 ceph-mon[33633]: pgmap v1257: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:43.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:43 smithi100 ceph-mon[30138]: pgmap v1257: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:43.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:43 smithi100 ceph-mon[39557]: pgmap v1257: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:44.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:44.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:45 smithi179 ceph-mon[33633]: pgmap v1258: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:04:45.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:45 smithi100 ceph-mon[39557]: pgmap v1258: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:04:45.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:45 smithi100 ceph-mon[30138]: pgmap v1258: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:04:47.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:47 smithi179 ceph-mon[33633]: pgmap v1259: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:04:47.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:47 smithi100 ceph-mon[39557]: pgmap v1259: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:04:47.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:47 smithi100 ceph-mon[30138]: pgmap v1259: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:04:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:49 smithi179 ceph-mon[33633]: pgmap v1260: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:49 smithi100 ceph-mon[30138]: pgmap v1260: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:49.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:49 smithi100 ceph-mon[39557]: pgmap v1260: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:04:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:04:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:04:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:51 smithi179 ceph-mon[33633]: pgmap v1261: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:04:51.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:51 smithi100 ceph-mon[39557]: pgmap v1261: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:04:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:51 smithi100 ceph-mon[30138]: pgmap v1261: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:04:53.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:04:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:04:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:53 smithi179 ceph-mon[33633]: pgmap v1262: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:53.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:53 smithi100 ceph-mon[39557]: pgmap v1262: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:53.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:53 smithi100 ceph-mon[30138]: pgmap v1262: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:54.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:04:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:55 smithi179 ceph-mon[33633]: pgmap v1263: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:55 smithi100 ceph-mon[39557]: pgmap v1263: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:55.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:55 smithi100 ceph-mon[30138]: pgmap v1263: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:04:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:57 smithi179 ceph-mon[33633]: pgmap v1264: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:57.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:57 smithi100 ceph-mon[39557]: pgmap v1264: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:57.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:57 smithi100 ceph-mon[30138]: pgmap v1264: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:04:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:58.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:04:59.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:04:59 smithi100 ceph-mon[30138]: pgmap v1265: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:59.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:04:59 smithi100 ceph-mon[39557]: pgmap v1265: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:04:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:04:59 smithi179 ceph-mon[33633]: pgmap v1265: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:05:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:05:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:05:01.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:01 smithi100 ceph-mon[30138]: pgmap v1266: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:01.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:01 smithi100 ceph-mon[39557]: pgmap v1266: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:01 smithi179 ceph-mon[33633]: pgmap v1266: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:03.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:05:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:05:03.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:03 smithi100 ceph-mon[30138]: pgmap v1267: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:03 smithi100 ceph-mon[39557]: pgmap v1267: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:03 smithi179 ceph-mon[33633]: pgmap v1267: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:04.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:04.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:05 smithi100 ceph-mon[30138]: pgmap v1268: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:05.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:05 smithi100 ceph-mon[39557]: pgmap v1268: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:05 smithi179 ceph-mon[33633]: pgmap v1268: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:07.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:07 smithi100 ceph-mon[30138]: pgmap v1269: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:05:07.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:07 smithi100 ceph-mon[39557]: pgmap v1269: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:05:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:07 smithi179 ceph-mon[33633]: pgmap v1269: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:05:09.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:09 smithi100 ceph-mon[30138]: pgmap v1270: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 7.2 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-14T18:05:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:09 smithi100 ceph-mon[39557]: pgmap v1270: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 7.2 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-14T18:05:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:09 smithi179 ceph-mon[33633]: pgmap v1270: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 7.2 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-14T18:05:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:05:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:05:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:05:11.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:11 smithi100 ceph-mon[30138]: pgmap v1271: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T18:05:11.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:11 smithi100 ceph-mon[39557]: pgmap v1271: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T18:05:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:11 smithi179 ceph-mon[33633]: pgmap v1271: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-14T18:05:13.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:05:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:05:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:13 smithi100 ceph-mon[30138]: pgmap v1272: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:13 smithi100 ceph-mon[39557]: pgmap v1272: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:13 smithi179 ceph-mon[33633]: pgmap v1272: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:14.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:14.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:15.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:15 smithi100 ceph-mon[30138]: pgmap v1273: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:15.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:15 smithi100 ceph-mon[39557]: pgmap v1273: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:15 smithi179 ceph-mon[33633]: pgmap v1273: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:17.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:17 smithi100 ceph-mon[30138]: pgmap v1274: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:17 smithi100 ceph-mon[39557]: pgmap v1274: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:17 smithi179 ceph-mon[33633]: pgmap v1274: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:19.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:19 smithi100 ceph-mon[30138]: pgmap v1275: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:19.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:19 smithi100 ceph-mon[39557]: pgmap v1275: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:19 smithi179 ceph-mon[33633]: pgmap v1275: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:05:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:05:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:05:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:05:21.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:21 smithi100 ceph-mon[30138]: pgmap v1276: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 31 KiB/s rd, 0 B/s wr, 50 op/s 2024-02-14T18:05:21.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:21 smithi100 ceph-mon[39557]: pgmap v1276: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 31 KiB/s rd, 0 B/s wr, 50 op/s 2024-02-14T18:05:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:21 smithi179 ceph-mon[33633]: pgmap v1276: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 31 KiB/s rd, 0 B/s wr, 50 op/s 2024-02-14T18:05:23.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:05:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:05:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:23 smithi100 ceph-mon[30138]: pgmap v1277: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 31 KiB/s rd, 0 B/s wr, 50 op/s 2024-02-14T18:05:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:23 smithi100 ceph-mon[39557]: pgmap v1277: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 31 KiB/s rd, 0 B/s wr, 50 op/s 2024-02-14T18:05:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:23 smithi179 ceph-mon[33633]: pgmap v1277: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 31 KiB/s rd, 0 B/s wr, 50 op/s 2024-02-14T18:05:24.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:25 smithi100 ceph-mon[30138]: pgmap v1278: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:25 smithi100 ceph-mon[39557]: pgmap v1278: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:25 smithi179 ceph-mon[33633]: pgmap v1278: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.588 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.589 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.590 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.591 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.592 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.593 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.594 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:25.595 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:05:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:27 smithi100 ceph-mon[30138]: pgmap v1279: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:27.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:27 smithi100 ceph-mon[39557]: pgmap v1279: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:27 smithi179 ceph-mon[33633]: pgmap v1279: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:28.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:28.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:29.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:29 smithi100 ceph-mon[30138]: pgmap v1280: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:29.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:29 smithi100 ceph-mon[39557]: pgmap v1280: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:29 smithi179 ceph-mon[33633]: pgmap v1280: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:05:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:05:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:05:31.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:31 smithi100 ceph-mon[30138]: pgmap v1281: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:31.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:31 smithi100 ceph-mon[39557]: pgmap v1281: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:31 smithi179 ceph-mon[33633]: pgmap v1281: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:33.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:05:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:05:33.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:33 smithi100 ceph-mon[30138]: pgmap v1282: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:33.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:33 smithi100 ceph-mon[39557]: pgmap v1282: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:33 smithi179 ceph-mon[33633]: pgmap v1282: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:35 smithi179 ceph-mon[33633]: pgmap v1283: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:35 smithi100 ceph-mon[30138]: pgmap v1283: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:35.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:35 smithi100 ceph-mon[39557]: pgmap v1283: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:36 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:05:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:36 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:05:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:36 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:05:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:36 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:05:36.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:36 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:05:36.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:36 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:05:36.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:36 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:05:36.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:36 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:05:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:36 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:05:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:36 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:05:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:36 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:05:36.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:36 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:05:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:37 smithi179 ceph-mon[33633]: pgmap v1284: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:37.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:37 smithi100 ceph-mon[30138]: pgmap v1284: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:37 smithi100 ceph-mon[39557]: pgmap v1284: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:39 smithi179 ceph-mon[33633]: pgmap v1285: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:39 smithi100 ceph-mon[30138]: pgmap v1285: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:39.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:39 smithi100 ceph-mon[39557]: pgmap v1285: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:05:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:05:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:05:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:41 smithi179 ceph-mon[33633]: pgmap v1286: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:41.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:41 smithi100 ceph-mon[30138]: pgmap v1286: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:41 smithi100 ceph-mon[39557]: pgmap v1286: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:43.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:05:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:05:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:43 smithi179 ceph-mon[33633]: pgmap v1287: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:43 smithi100 ceph-mon[30138]: pgmap v1287: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:43 smithi100 ceph-mon[39557]: pgmap v1287: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:44.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:45 smithi179 ceph-mon[33633]: pgmap v1288: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:45.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:45 smithi100 ceph-mon[30138]: pgmap v1288: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:45.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:45 smithi100 ceph-mon[39557]: pgmap v1288: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:46 smithi179 ceph-mon[33633]: pgmap v1289: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:05:46.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:46 smithi100 ceph-mon[30138]: pgmap v1289: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:05:46.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:46 smithi100 ceph-mon[39557]: pgmap v1289: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:05:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:48 smithi179 ceph-mon[33633]: pgmap v1290: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:48 smithi100 ceph-mon[30138]: pgmap v1290: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:49.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:48 smithi100 ceph-mon[39557]: pgmap v1290: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:05:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:05:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:05:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:50 smithi179 ceph-mon[33633]: pgmap v1291: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:50 smithi100 ceph-mon[30138]: pgmap v1291: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:51.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:50 smithi100 ceph-mon[39557]: pgmap v1291: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:52.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:05:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:05:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:52 smithi179 ceph-mon[33633]: pgmap v1292: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:52 smithi100 ceph-mon[30138]: pgmap v1292: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:53.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:52 smithi100 ceph-mon[39557]: pgmap v1292: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:54.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:05:55.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:55 smithi179 ceph-mon[33633]: pgmap v1293: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:55.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:55 smithi100 ceph-mon[30138]: pgmap v1293: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:55.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:55 smithi100 ceph-mon[39557]: pgmap v1293: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:05:57.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:57 smithi179 ceph-mon[33633]: pgmap v1294: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:05:57.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:57 smithi100 ceph-mon[30138]: pgmap v1294: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:05:57.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:57 smithi100 ceph-mon[39557]: pgmap v1294: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:05:58.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:58.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:05:59.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:05:59 smithi179 ceph-mon[33633]: pgmap v1295: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:59.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:05:59 smithi100 ceph-mon[39557]: pgmap v1295: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:05:59.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:05:59 smithi100 ceph-mon[30138]: pgmap v1295: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:06:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:06:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:06:01.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:01 smithi179 ceph-mon[33633]: pgmap v1296: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:01.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:01 smithi100 ceph-mon[39557]: pgmap v1296: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:01.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:01 smithi100 ceph-mon[30138]: pgmap v1296: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:03.025 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:06:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:06:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:03 smithi179 ceph-mon[33633]: pgmap v1297: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:03.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:03 smithi100 ceph-mon[30138]: pgmap v1297: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:03 smithi100 ceph-mon[39557]: pgmap v1297: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:04.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:04.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:04.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:05.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:05 smithi179 ceph-mon[33633]: pgmap v1298: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:05 smithi100 ceph-mon[30138]: pgmap v1298: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:05.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:05 smithi100 ceph-mon[39557]: pgmap v1298: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:07.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:07 smithi179 ceph-mon[33633]: pgmap v1299: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:07.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:07 smithi100 ceph-mon[39557]: pgmap v1299: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:07.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:07 smithi100 ceph-mon[30138]: pgmap v1299: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:09.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:09 smithi179 ceph-mon[33633]: pgmap v1300: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:09.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:09 smithi100 ceph-mon[30138]: pgmap v1300: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:09 smithi100 ceph-mon[39557]: pgmap v1300: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:06:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:06:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:06:11.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:11 smithi179 ceph-mon[33633]: pgmap v1301: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:11.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:11 smithi100 ceph-mon[30138]: pgmap v1301: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:11.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:11 smithi100 ceph-mon[39557]: pgmap v1301: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:13.043 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:06:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:06:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:13 smithi179 ceph-mon[33633]: pgmap v1302: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:13 smithi100 ceph-mon[30138]: pgmap v1302: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:13 smithi100 ceph-mon[39557]: pgmap v1302: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:14.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:14.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:14.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:15.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:15 smithi179 ceph-mon[33633]: pgmap v1303: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:15.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:15 smithi100 ceph-mon[30138]: pgmap v1303: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:15.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:15 smithi100 ceph-mon[39557]: pgmap v1303: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:17.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:17 smithi179 ceph-mon[33633]: pgmap v1304: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:17.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:17 smithi100 ceph-mon[30138]: pgmap v1304: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:17.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:17 smithi100 ceph-mon[39557]: pgmap v1304: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:19.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:19 smithi179 ceph-mon[33633]: pgmap v1305: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:19.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:19 smithi100 ceph-mon[39557]: pgmap v1305: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:19.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:19 smithi100 ceph-mon[30138]: pgmap v1305: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:06:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:06:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:06:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:21 smithi179 ceph-mon[33633]: pgmap v1306: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:21.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:21 smithi100 ceph-mon[30138]: pgmap v1306: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:21.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:21 smithi100 ceph-mon[39557]: pgmap v1306: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:23.062 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:06:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:06:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:23 smithi179 ceph-mon[33633]: pgmap v1307: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:23 smithi100 ceph-mon[39557]: pgmap v1307: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:23 smithi100 ceph-mon[30138]: pgmap v1307: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:24.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:25 smithi179 ceph-mon[33633]: pgmap v1308: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:25 smithi100 ceph-mon[30138]: pgmap v1308: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:25 smithi100 ceph-mon[39557]: pgmap v1308: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:27 smithi100 ceph-mon[30138]: pgmap v1309: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:27.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:27 smithi100 ceph-mon[39557]: pgmap v1309: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:27 smithi179 ceph-mon[33633]: pgmap v1309: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:28.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:28.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:29.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:29 smithi100 ceph-mon[30138]: pgmap v1310: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:29.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:29 smithi100 ceph-mon[39557]: pgmap v1310: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:29 smithi179 ceph-mon[33633]: pgmap v1310: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:06:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:06:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:06:31.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:31 smithi100 ceph-mon[30138]: pgmap v1311: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:31 smithi100 ceph-mon[39557]: pgmap v1311: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:31 smithi179 ceph-mon[33633]: pgmap v1311: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:33.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:06:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:06:33.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:33 smithi100 ceph-mon[30138]: pgmap v1312: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:33.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:33 smithi100 ceph-mon[39557]: pgmap v1312: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:33 smithi179 ceph-mon[33633]: pgmap v1312: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:34.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:34.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:35 smithi100 ceph-mon[30138]: pgmap v1313: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:35.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:35 smithi100 ceph-mon[39557]: pgmap v1313: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:35 smithi179 ceph-mon[33633]: pgmap v1313: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:36.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:36 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:06:36.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:36 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:06:36.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:36 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:06:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:37 smithi100 ceph-mon[30138]: pgmap v1314: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:37 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:06:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:37 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:06:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:37 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:06:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:37 smithi100 ceph-mon[39557]: pgmap v1314: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:37 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:06:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:37 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:06:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:37 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:06:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:37 smithi179 ceph-mon[33633]: pgmap v1314: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:37 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:06:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:37 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:06:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:37 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:06:39.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:39 smithi100 ceph-mon[39557]: pgmap v1315: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:39.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:39 smithi100 ceph-mon[30138]: pgmap v1315: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:39 smithi179 ceph-mon[33633]: pgmap v1315: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:06:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:06:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:06:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:41 smithi100 ceph-mon[30138]: pgmap v1316: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:41.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:41 smithi100 ceph-mon[39557]: pgmap v1316: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:41.463 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:41 smithi179 ceph-mon[33633]: pgmap v1316: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:41.821 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:06:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=cleanup t=2024-02-14T18:06:41.462520421Z level=info msg="Completed cleanup jobs" duration=6.410912ms 2024-02-14T18:06:43.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:06:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:06:43.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:43 smithi100 ceph-mon[30138]: pgmap v1317: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:43.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:43.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:43 smithi100 ceph-mon[39557]: pgmap v1317: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:43 smithi179 ceph-mon[33633]: pgmap v1317: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:45.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:45 smithi100 ceph-mon[39557]: pgmap v1318: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:45.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:45 smithi100 ceph-mon[30138]: pgmap v1318: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:45 smithi179 ceph-mon[33633]: pgmap v1318: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:47.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:47 smithi100 ceph-mon[30138]: pgmap v1319: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:47.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:47 smithi100 ceph-mon[39557]: pgmap v1319: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:47 smithi179 ceph-mon[33633]: pgmap v1319: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:49.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:49 smithi100 ceph-mon[39557]: pgmap v1320: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:49.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:49 smithi100 ceph-mon[30138]: pgmap v1320: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:49 smithi179 ceph-mon[33633]: pgmap v1320: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:06:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:06:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:06:51.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:51 smithi100 ceph-mon[30138]: pgmap v1321: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:51.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:51 smithi100 ceph-mon[39557]: pgmap v1321: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:51 smithi179 ceph-mon[33633]: pgmap v1321: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:53.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:06:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:06:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:53 smithi100 ceph-mon[30138]: pgmap v1322: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:53.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:53 smithi100 ceph-mon[39557]: pgmap v1322: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:53 smithi179 ceph-mon[33633]: pgmap v1322: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:54.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:06:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:55 smithi100 ceph-mon[39557]: pgmap v1323: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:55.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:55 smithi100 ceph-mon[30138]: pgmap v1323: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:55 smithi179 ceph-mon[33633]: pgmap v1323: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:06:57.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:57 smithi100 ceph-mon[39557]: pgmap v1324: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:57.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:57 smithi100 ceph-mon[30138]: pgmap v1324: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:57 smithi179 ceph-mon[33633]: pgmap v1324: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:06:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:06:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:06:59 smithi179 ceph-mon[33633]: pgmap v1325: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:06:59 smithi100 ceph-mon[30138]: pgmap v1325: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:06:59.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:06:59 smithi100 ceph-mon[39557]: pgmap v1325: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:07:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:07:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:07:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:01 smithi179 ceph-mon[33633]: pgmap v1326: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:01 smithi100 ceph-mon[30138]: pgmap v1326: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:01.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:01 smithi100 ceph-mon[39557]: pgmap v1326: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:03.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:07:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:07:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:03 smithi179 ceph-mon[33633]: pgmap v1327: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:03 smithi100 ceph-mon[30138]: pgmap v1327: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:03 smithi100 ceph-mon[39557]: pgmap v1327: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:05 smithi179 ceph-mon[33633]: pgmap v1328: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:05.599 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.599 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.599 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.599 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.599 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.599 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.600 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.601 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.602 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.603 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.604 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.605 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.606 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.606 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.606 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.606 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.606 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.606 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.606 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.606 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:07:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:05 smithi100 ceph-mon[39557]: pgmap v1328: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:05 smithi100 ceph-mon[30138]: pgmap v1328: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:07 smithi179 ceph-mon[33633]: pgmap v1329: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:07.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:07 smithi100 ceph-mon[39557]: pgmap v1329: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:07 smithi100 ceph-mon[30138]: pgmap v1329: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:09 smithi179 ceph-mon[33633]: pgmap v1330: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:09.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:09 smithi100 ceph-mon[39557]: pgmap v1330: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:09.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:09 smithi100 ceph-mon[30138]: pgmap v1330: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:07:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:07:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:07:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:11 smithi179 ceph-mon[33633]: pgmap v1331: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:11.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:11 smithi100 ceph-mon[39557]: pgmap v1331: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:11.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:11 smithi100 ceph-mon[30138]: pgmap v1331: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:13.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:07:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:07:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:13 smithi179 ceph-mon[33633]: pgmap v1332: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:13.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:13 smithi100 ceph-mon[39557]: pgmap v1332: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:13.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:13 smithi100 ceph-mon[30138]: pgmap v1332: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:14.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:15 smithi179 ceph-mon[33633]: pgmap v1333: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:15.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:15 smithi100 ceph-mon[39557]: pgmap v1333: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:15 smithi100 ceph-mon[30138]: pgmap v1333: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:17 smithi179 ceph-mon[33633]: pgmap v1334: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:17 smithi100 ceph-mon[30138]: pgmap v1334: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:17.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:17 smithi100 ceph-mon[39557]: pgmap v1334: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:18.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:18 smithi179 ceph-mon[33633]: pgmap v1335: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:18.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:18 smithi100 ceph-mon[30138]: pgmap v1335: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:18.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:18 smithi100 ceph-mon[39557]: pgmap v1335: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:07:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:07:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:07:21.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:21 smithi179 ceph-mon[33633]: pgmap v1336: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:21.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:21 smithi100 ceph-mon[30138]: pgmap v1336: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:21.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:21 smithi100 ceph-mon[39557]: pgmap v1336: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:23.037 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:07:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:07:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:23 smithi179 ceph-mon[33633]: pgmap v1337: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:23 smithi100 ceph-mon[30138]: pgmap v1337: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:23 smithi100 ceph-mon[39557]: pgmap v1337: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:24.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:24.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:25 smithi179 ceph-mon[33633]: pgmap v1338: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:25 smithi100 ceph-mon[30138]: pgmap v1338: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:25 smithi100 ceph-mon[39557]: pgmap v1338: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:27.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:27 smithi179 ceph-mon[33633]: pgmap v1339: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:27 smithi100 ceph-mon[30138]: pgmap v1339: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:27.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:27 smithi100 ceph-mon[39557]: pgmap v1339: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:28.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:28.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:28.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:29.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:29 smithi179 ceph-mon[33633]: pgmap v1340: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:29.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:29 smithi100 ceph-mon[30138]: pgmap v1340: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:29.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:29 smithi100 ceph-mon[39557]: pgmap v1340: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:07:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:07:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:07:31.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:31 smithi179 ceph-mon[33633]: pgmap v1341: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:31.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:31 smithi100 ceph-mon[30138]: pgmap v1341: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:31.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:31 smithi100 ceph-mon[39557]: pgmap v1341: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:33.063 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:07:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:07:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:33 smithi179 ceph-mon[33633]: pgmap v1342: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:33.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:33 smithi100 ceph-mon[30138]: pgmap v1342: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:33.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:33 smithi100 ceph-mon[39557]: pgmap v1342: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:34.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:34.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:34.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:35.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:35 smithi179 ceph-mon[33633]: pgmap v1343: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:35.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:35 smithi100 ceph-mon[30138]: pgmap v1343: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:35.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:35 smithi100 ceph-mon[39557]: pgmap v1343: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:37 smithi100 ceph-mon[30138]: pgmap v1344: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:37 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:07:37.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:37 smithi100 ceph-mon[39557]: pgmap v1344: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:37 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:07:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:37 smithi179 ceph-mon[33633]: pgmap v1344: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:37 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:07:38.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:38 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:07:38.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:38 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:07:38.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:38 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:07:38.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:38 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:07:38.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:38 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:07:38.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:38 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:07:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:38 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:07:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:38 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:07:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:38 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:07:39.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:39 smithi100 ceph-mon[39557]: pgmap v1345: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:39.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:39 smithi100 ceph-mon[30138]: pgmap v1345: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:39 smithi179 ceph-mon[33633]: pgmap v1345: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:07:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:07:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:07:41.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:41 smithi100 ceph-mon[39557]: pgmap v1346: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:41.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:41 smithi100 ceph-mon[30138]: pgmap v1346: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:41 smithi179 ceph-mon[33633]: pgmap v1346: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:43.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:07:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:07:43.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:43 smithi100 ceph-mon[30138]: pgmap v1347: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:43.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:43 smithi100 ceph-mon[39557]: pgmap v1347: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:43.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:43 smithi179 ceph-mon[33633]: pgmap v1347: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:44.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:44.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:45.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:45 smithi100 ceph-mon[39557]: pgmap v1348: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:45.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:45 smithi100 ceph-mon[30138]: pgmap v1348: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:45 smithi179 ceph-mon[33633]: pgmap v1348: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:47.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:47 smithi100 ceph-mon[39557]: pgmap v1349: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:47.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:47 smithi100 ceph-mon[30138]: pgmap v1349: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:47 smithi179 ceph-mon[33633]: pgmap v1349: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:49.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:49 smithi100 ceph-mon[39557]: pgmap v1350: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:49 smithi100 ceph-mon[30138]: pgmap v1350: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:49 smithi179 ceph-mon[33633]: pgmap v1350: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:07:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:07:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:07:51.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:51 smithi100 ceph-mon[39557]: pgmap v1351: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:51 smithi100 ceph-mon[30138]: pgmap v1351: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:51 smithi179 ceph-mon[33633]: pgmap v1351: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:53.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:07:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:07:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:53 smithi100 ceph-mon[30138]: pgmap v1352: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:53 smithi100 ceph-mon[39557]: pgmap v1352: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:53 smithi179 ceph-mon[33633]: pgmap v1352: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:07:55.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:55 smithi100 ceph-mon[30138]: pgmap v1353: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:55 smithi100 ceph-mon[39557]: pgmap v1353: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:55 smithi179 ceph-mon[33633]: pgmap v1353: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:07:57.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:57 smithi100 ceph-mon[30138]: pgmap v1354: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:57.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:57 smithi100 ceph-mon[39557]: pgmap v1354: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:57 smithi179 ceph-mon[33633]: pgmap v1354: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:07:58.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:07:59.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:07:59 smithi100 ceph-mon[39557]: pgmap v1355: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:59.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:07:59 smithi100 ceph-mon[30138]: pgmap v1355: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:07:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:07:59 smithi179 ceph-mon[33633]: pgmap v1355: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:08:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:08:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:08:01.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:01 smithi100 ceph-mon[30138]: pgmap v1356: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:01.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:01 smithi100 ceph-mon[39557]: pgmap v1356: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:01 smithi179 ceph-mon[33633]: pgmap v1356: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:03.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:08:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:08:03.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:03 smithi100 ceph-mon[30138]: pgmap v1357: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:03 smithi100 ceph-mon[39557]: pgmap v1357: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:03 smithi179 ceph-mon[33633]: pgmap v1357: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:04.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:04.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:05 smithi100 ceph-mon[30138]: pgmap v1358: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:05.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:05 smithi100 ceph-mon[39557]: pgmap v1358: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:05 smithi179 ceph-mon[33633]: pgmap v1358: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:07.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:07 smithi100 ceph-mon[30138]: pgmap v1359: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:07 smithi100 ceph-mon[39557]: pgmap v1359: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:07 smithi179 ceph-mon[33633]: pgmap v1359: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:09.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:09 smithi100 ceph-mon[30138]: pgmap v1360: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:09.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:09 smithi100 ceph-mon[39557]: pgmap v1360: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:09 smithi179 ceph-mon[33633]: pgmap v1360: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:08:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:08:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:08:11.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:11 smithi100 ceph-mon[30138]: pgmap v1361: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:11.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:11 smithi100 ceph-mon[39557]: pgmap v1361: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:11 smithi179 ceph-mon[33633]: pgmap v1361: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:13.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:08:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:08:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:13 smithi100 ceph-mon[39557]: pgmap v1362: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:13 smithi100 ceph-mon[30138]: pgmap v1362: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:13 smithi179 ceph-mon[33633]: pgmap v1362: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:14.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:15 smithi179 ceph-mon[33633]: pgmap v1363: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:15 smithi100 ceph-mon[30138]: pgmap v1363: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:15 smithi100 ceph-mon[39557]: pgmap v1363: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:17 smithi179 ceph-mon[33633]: pgmap v1364: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:17.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:17 smithi100 ceph-mon[39557]: pgmap v1364: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:17 smithi100 ceph-mon[30138]: pgmap v1364: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:19 smithi179 ceph-mon[33633]: pgmap v1365: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:19.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:19 smithi100 ceph-mon[39557]: pgmap v1365: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:19.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:19 smithi100 ceph-mon[30138]: pgmap v1365: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:08:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:08:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:08:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:21 smithi179 ceph-mon[33633]: pgmap v1366: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:21.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:21 smithi100 ceph-mon[30138]: pgmap v1366: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:21.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:21 smithi100 ceph-mon[39557]: pgmap v1366: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:23.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:08:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:08:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:23 smithi179 ceph-mon[33633]: pgmap v1367: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:23.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:23 smithi100 ceph-mon[39557]: pgmap v1367: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:23.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:23 smithi100 ceph-mon[30138]: pgmap v1367: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:24.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:24.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:25 smithi179 ceph-mon[33633]: pgmap v1368: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:25.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:25 smithi100 ceph-mon[30138]: pgmap v1368: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:25.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:25 smithi100 ceph-mon[39557]: pgmap v1368: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:27 smithi179 ceph-mon[33633]: pgmap v1369: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:27.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:27 smithi100 ceph-mon[30138]: pgmap v1369: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:27.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:27 smithi100 ceph-mon[39557]: pgmap v1369: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:29.573 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:29 smithi179 ceph-mon[33633]: pgmap v1370: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:29 smithi100 ceph-mon[30138]: pgmap v1370: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:29.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:29 smithi100 ceph-mon[39557]: pgmap v1370: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:08:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:08:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:08:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:31 smithi179 ceph-mon[33633]: pgmap v1371: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:31.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:31 smithi100 ceph-mon[39557]: pgmap v1371: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:31.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:31 smithi100 ceph-mon[30138]: pgmap v1371: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:33.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:08:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:08:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:33 smithi179 ceph-mon[33633]: pgmap v1372: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:33 smithi100 ceph-mon[39557]: pgmap v1372: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:33.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:33 smithi100 ceph-mon[30138]: pgmap v1372: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:35 smithi179 ceph-mon[33633]: pgmap v1373: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:35.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:35 smithi100 ceph-mon[39557]: pgmap v1373: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:35 smithi100 ceph-mon[30138]: pgmap v1373: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:37 smithi179 ceph-mon[33633]: pgmap v1374: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:37.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:37 smithi100 ceph-mon[30138]: pgmap v1374: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:37 smithi100 ceph-mon[39557]: pgmap v1374: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:38.512 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:38 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:08:38.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:38 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:08:38.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:38 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:08:39.432 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:39 smithi179 ceph-mon[33633]: pgmap v1375: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:39.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:39 smithi100 ceph-mon[39557]: pgmap v1375: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:39 smithi100 ceph-mon[30138]: pgmap v1375: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:40.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:40 smithi179 ceph-mon[33633]: pgmap v1376: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:40.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:40 smithi100 ceph-mon[30138]: pgmap v1376: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:40.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:40 smithi100 ceph-mon[39557]: pgmap v1376: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:40.681 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:08:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:08:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:08:41.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:41.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:41 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:41.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:41.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:41 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:41.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:41.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:41 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:43.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:08:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:08:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:42 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:42 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:42 smithi179 ceph-mon[33633]: pgmap v1377: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:08:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:08:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:42 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:43.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:42 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:43.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[30138]: pgmap v1377: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[39557]: pgmap v1377: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:08:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:08:43.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:08:43.151 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:42 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:44.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:43 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:44.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:43 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:44.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:43 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:45.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:44 smithi100 ceph-mon[30138]: pgmap v1378: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:45.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:44 smithi100 ceph-mon[39557]: pgmap v1378: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:45.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:44 smithi179 ceph-mon[33633]: pgmap v1378: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:45.611 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.611 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.611 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.612 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.613 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.614 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.615 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.616 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.617 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.617 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.617 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.617 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.617 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.617 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.617 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.617 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.617 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.617 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.618 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.618 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.618 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.618 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.618 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.619 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.620 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.620 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.620 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:45.620 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:08:47.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:47 smithi179 ceph-mon[33633]: pgmap v1379: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:47.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:47 smithi100 ceph-mon[30138]: pgmap v1379: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:47.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:47 smithi100 ceph-mon[39557]: pgmap v1379: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:08:49.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:49 smithi179 ceph-mon[33633]: pgmap v1380: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:49.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:49 smithi100 ceph-mon[30138]: pgmap v1380: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:49.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:49 smithi100 ceph-mon[39557]: pgmap v1380: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:08:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:08:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:08:51.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:51 smithi179 ceph-mon[33633]: pgmap v1381: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:51.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:51 smithi100 ceph-mon[39557]: pgmap v1381: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:51.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:51 smithi100 ceph-mon[30138]: pgmap v1381: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:53.069 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:08:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:08:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:53 smithi179 ceph-mon[33633]: pgmap v1382: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:53.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:53 smithi100 ceph-mon[30138]: pgmap v1382: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:53.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:53 smithi100 ceph-mon[39557]: pgmap v1382: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:54.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:54.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:08:55.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:55 smithi100 ceph-mon[30138]: pgmap v1383: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:55.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:55 smithi100 ceph-mon[39557]: pgmap v1383: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:55 smithi179 ceph-mon[33633]: pgmap v1383: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:57.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:57 smithi100 ceph-mon[30138]: pgmap v1384: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:57.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:57 smithi100 ceph-mon[39557]: pgmap v1384: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:57 smithi179 ceph-mon[33633]: pgmap v1384: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:08:58.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:58.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:08:59.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:08:59 smithi100 ceph-mon[30138]: pgmap v1385: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:59.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:08:59 smithi100 ceph-mon[39557]: pgmap v1385: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:08:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:08:59 smithi179 ceph-mon[33633]: pgmap v1385: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:09:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:09:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:09:01.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:01 smithi100 ceph-mon[30138]: pgmap v1386: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:01.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:01 smithi100 ceph-mon[39557]: pgmap v1386: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:01 smithi179 ceph-mon[33633]: pgmap v1386: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:03.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:09:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:09:03.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:03 smithi100 ceph-mon[30138]: pgmap v1387: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:03.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:03 smithi100 ceph-mon[39557]: pgmap v1387: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:03 smithi179 ceph-mon[33633]: pgmap v1387: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:04.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:04.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:05.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:05 smithi100 ceph-mon[39557]: pgmap v1388: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:05 smithi100 ceph-mon[30138]: pgmap v1388: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:05 smithi179 ceph-mon[33633]: pgmap v1388: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:07.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:07 smithi100 ceph-mon[30138]: pgmap v1389: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:07.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:07 smithi100 ceph-mon[39557]: pgmap v1389: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:07 smithi179 ceph-mon[33633]: pgmap v1389: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:09.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:09 smithi100 ceph-mon[30138]: pgmap v1390: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:09 smithi100 ceph-mon[39557]: pgmap v1390: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:09 smithi179 ceph-mon[33633]: pgmap v1390: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:09:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:09:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:09:11.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:11 smithi100 ceph-mon[30138]: pgmap v1391: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:11.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:11 smithi100 ceph-mon[39557]: pgmap v1391: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:11 smithi179 ceph-mon[33633]: pgmap v1391: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:13.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:09:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:09:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:13 smithi100 ceph-mon[30138]: pgmap v1392: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:13 smithi100 ceph-mon[39557]: pgmap v1392: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:13.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:13 smithi179 ceph-mon[33633]: pgmap v1392: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:14.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:14.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:15.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:15 smithi100 ceph-mon[30138]: pgmap v1393: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:15.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:15 smithi100 ceph-mon[39557]: pgmap v1393: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:15 smithi179 ceph-mon[33633]: pgmap v1393: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:17.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:17 smithi100 ceph-mon[30138]: pgmap v1394: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:17.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:17 smithi100 ceph-mon[39557]: pgmap v1394: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:17 smithi179 ceph-mon[33633]: pgmap v1394: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:19.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:19 smithi100 ceph-mon[30138]: pgmap v1395: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:19.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:19 smithi100 ceph-mon[39557]: pgmap v1395: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:19 smithi179 ceph-mon[33633]: pgmap v1395: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:20.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:09:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:09:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:09:21.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:21 smithi100 ceph-mon[39557]: pgmap v1396: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:21.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:21 smithi100 ceph-mon[30138]: pgmap v1396: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:21 smithi179 ceph-mon[33633]: pgmap v1396: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:23.071 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:09:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:09:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:23 smithi100 ceph-mon[30138]: pgmap v1397: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:23 smithi100 ceph-mon[39557]: pgmap v1397: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:23 smithi179 ceph-mon[33633]: pgmap v1397: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:24.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:25 smithi100 ceph-mon[30138]: pgmap v1398: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:25 smithi100 ceph-mon[39557]: pgmap v1398: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:25 smithi179 ceph-mon[33633]: pgmap v1398: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:27 smithi100 ceph-mon[30138]: pgmap v1399: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:09:27.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:27 smithi100 ceph-mon[39557]: pgmap v1399: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:09:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:27 smithi179 ceph-mon[33633]: pgmap v1399: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:09:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:29 smithi179 ceph-mon[33633]: pgmap v1400: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:29 smithi100 ceph-mon[30138]: pgmap v1400: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:29.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:29 smithi100 ceph-mon[39557]: pgmap v1400: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:09:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:09:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:09:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:31 smithi179 ceph-mon[33633]: pgmap v1401: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:31.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:31 smithi100 ceph-mon[39557]: pgmap v1401: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:31.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:31 smithi100 ceph-mon[30138]: pgmap v1401: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:33.159 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:09:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:09:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:33 smithi179 ceph-mon[33633]: pgmap v1402: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:33.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:33 smithi100 ceph-mon[30138]: pgmap v1402: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:33 smithi100 ceph-mon[39557]: pgmap v1402: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:35 smithi179 ceph-mon[33633]: pgmap v1403: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:35 smithi100 ceph-mon[30138]: pgmap v1403: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:35 smithi100 ceph-mon[39557]: pgmap v1403: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:37 smithi179 ceph-mon[33633]: pgmap v1404: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:09:37.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:37 smithi100 ceph-mon[30138]: pgmap v1404: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:09:37.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:37 smithi100 ceph-mon[39557]: pgmap v1404: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:09:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:39 smithi179 ceph-mon[33633]: pgmap v1405: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:39 smithi100 ceph-mon[30138]: pgmap v1405: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:39.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:39 smithi100 ceph-mon[39557]: pgmap v1405: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:09:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:09:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:09:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:41 smithi179 ceph-mon[33633]: pgmap v1406: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:41.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:41 smithi100 ceph-mon[30138]: pgmap v1406: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:41 smithi100 ceph-mon[39557]: pgmap v1406: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:43.180 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:09:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:09:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:43 smithi179 ceph-mon[33633]: pgmap v1407: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:09:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:09:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:09:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:43 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:09:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[30138]: pgmap v1407: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:09:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:09:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:09:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:09:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[39557]: pgmap v1407: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:09:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:09:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:09:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:43 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:09:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:45 smithi179 ceph-mon[33633]: pgmap v1408: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:45.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:45 smithi100 ceph-mon[39557]: pgmap v1408: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:45.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:45 smithi100 ceph-mon[30138]: pgmap v1408: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:47 smithi179 ceph-mon[33633]: pgmap v1409: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:09:47.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:47 smithi100 ceph-mon[30138]: pgmap v1409: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:09:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:47 smithi100 ceph-mon[39557]: pgmap v1409: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:09:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:49 smithi179 ceph-mon[33633]: pgmap v1410: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:49 smithi100 ceph-mon[30138]: pgmap v1410: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:49.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:49 smithi100 ceph-mon[39557]: pgmap v1410: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:09:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:09:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:09:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:51 smithi179 ceph-mon[33633]: pgmap v1411: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:51 smithi100 ceph-mon[30138]: pgmap v1411: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:51 smithi100 ceph-mon[39557]: pgmap v1411: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:53.200 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:09:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:09:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:53 smithi179 ceph-mon[33633]: pgmap v1412: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:53 smithi100 ceph-mon[30138]: pgmap v1412: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:53.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:53 smithi100 ceph-mon[39557]: pgmap v1412: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:54.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:09:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:55 smithi179 ceph-mon[33633]: pgmap v1413: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:55 smithi100 ceph-mon[30138]: pgmap v1413: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:55 smithi100 ceph-mon[39557]: pgmap v1413: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:57 smithi179 ceph-mon[33633]: pgmap v1414: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:57.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:57 smithi100 ceph-mon[30138]: pgmap v1414: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:57.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:57 smithi100 ceph-mon[39557]: pgmap v1414: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:09:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:09:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:09:59 smithi179 ceph-mon[33633]: pgmap v1415: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:09:59 smithi100 ceph-mon[30138]: pgmap v1415: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:09:59.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:09:59 smithi100 ceph-mon[39557]: pgmap v1415: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[39557]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-02-14T18:10:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[39557]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-02-14T18:10:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[39557]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:10:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[39557]: application not enabled on pool 'WatchNotifysmithi100-80894-1' 2024-02-14T18:10:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[39557]: application not enabled on pool 'poolrace.1804289383' 2024-02-14T18:10:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[39557]: application not enabled on pool 'poolrace2.846930886' 2024-02-14T18:10:00.493 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[39557]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:10:00.494 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[39557]: pgmap v1416: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[30138]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-02-14T18:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[30138]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-02-14T18:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[30138]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[30138]: application not enabled on pool 'WatchNotifysmithi100-80894-1' 2024-02-14T18:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[30138]: application not enabled on pool 'poolrace.1804289383' 2024-02-14T18:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[30138]: application not enabled on pool 'poolrace2.846930886' 2024-02-14T18:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[30138]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:10:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-mon[30138]: pgmap v1416: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:00 smithi179 ceph-mon[33633]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-02-14T18:10:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:00 smithi179 ceph-mon[33633]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-02-14T18:10:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:00 smithi179 ceph-mon[33633]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:10:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:00 smithi179 ceph-mon[33633]: application not enabled on pool 'WatchNotifysmithi100-80894-1' 2024-02-14T18:10:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:00 smithi179 ceph-mon[33633]: application not enabled on pool 'poolrace.1804289383' 2024-02-14T18:10:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:00 smithi179 ceph-mon[33633]: application not enabled on pool 'poolrace2.846930886' 2024-02-14T18:10:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:00 smithi179 ceph-mon[33633]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:10:00.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:00 smithi179 ceph-mon[33633]: pgmap v1416: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:10:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:10:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:10:03.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:10:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:10:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:03 smithi179 ceph-mon[33633]: pgmap v1417: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:03.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:03 smithi100 ceph-mon[30138]: pgmap v1417: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:03.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:03 smithi100 ceph-mon[39557]: pgmap v1417: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:04.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:04.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:05.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:05 smithi100 ceph-mon[30138]: pgmap v1418: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:05.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:05 smithi100 ceph-mon[39557]: pgmap v1418: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:05 smithi179 ceph-mon[33633]: pgmap v1418: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:07.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:07 smithi100 ceph-mon[30138]: pgmap v1419: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:07.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:07 smithi100 ceph-mon[39557]: pgmap v1419: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:07 smithi179 ceph-mon[33633]: pgmap v1419: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:09.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:09 smithi100 ceph-mon[30138]: pgmap v1420: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:09 smithi100 ceph-mon[39557]: pgmap v1420: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:09 smithi179 ceph-mon[33633]: pgmap v1420: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:10.583 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:10:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:10:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:10:11.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:11 smithi100 ceph-mon[39557]: pgmap v1421: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:11.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:11 smithi100 ceph-mon[30138]: pgmap v1421: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:11 smithi179 ceph-mon[33633]: pgmap v1421: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:13.115 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:10:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:10:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:13 smithi100 ceph-mon[39557]: pgmap v1422: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:13 smithi100 ceph-mon[30138]: pgmap v1422: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:13 smithi179 ceph-mon[33633]: pgmap v1422: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:14.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:14.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:15.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:15 smithi100 ceph-mon[39557]: pgmap v1423: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:15 smithi100 ceph-mon[30138]: pgmap v1423: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:15 smithi179 ceph-mon[33633]: pgmap v1423: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:17 smithi100 ceph-mon[39557]: pgmap v1424: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:17.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:17 smithi100 ceph-mon[30138]: pgmap v1424: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:17 smithi179 ceph-mon[33633]: pgmap v1424: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:19.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:19 smithi100 ceph-mon[39557]: pgmap v1425: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:19.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:19 smithi100 ceph-mon[30138]: pgmap v1425: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:19 smithi179 ceph-mon[33633]: pgmap v1425: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:20.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:10:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:10:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:10:21.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:21 smithi100 ceph-mon[39557]: pgmap v1426: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:21.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:21 smithi100 ceph-mon[30138]: pgmap v1426: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:21 smithi179 ceph-mon[33633]: pgmap v1426: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:23.138 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:10:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:10:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:23 smithi100 ceph-mon[39557]: pgmap v1427: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:23 smithi100 ceph-mon[30138]: pgmap v1427: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:23 smithi179 ceph-mon[33633]: pgmap v1427: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:24.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:24.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:25.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:25 smithi100 ceph-mon[30138]: pgmap v1428: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:25.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:25 smithi100 ceph-mon[39557]: pgmap v1428: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:25 smithi179 ceph-mon[33633]: pgmap v1428: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.623 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.624 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.625 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.626 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.627 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.628 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.629 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:25.630 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:10:27.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:27 smithi100 ceph-mon[30138]: pgmap v1429: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:27.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:27 smithi100 ceph-mon[39557]: pgmap v1429: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:27.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:27 smithi179 ceph-mon[33633]: pgmap v1429: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:28.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:29 smithi179 ceph-mon[33633]: pgmap v1430: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:29 smithi100 ceph-mon[30138]: pgmap v1430: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:29.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:29 smithi100 ceph-mon[39557]: pgmap v1430: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:10:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:10:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:10:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:31 smithi179 ceph-mon[33633]: pgmap v1431: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:31.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:31 smithi100 ceph-mon[30138]: pgmap v1431: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:31.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:31 smithi100 ceph-mon[39557]: pgmap v1431: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:33.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:10:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:10:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:33 smithi179 ceph-mon[33633]: pgmap v1432: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:33 smithi100 ceph-mon[39557]: pgmap v1432: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:33.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:33 smithi100 ceph-mon[30138]: pgmap v1432: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:35 smithi179 ceph-mon[33633]: pgmap v1433: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:35.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:35 smithi100 ceph-mon[39557]: pgmap v1433: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:35.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:35 smithi100 ceph-mon[30138]: pgmap v1433: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:37 smithi179 ceph-mon[33633]: pgmap v1434: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:37 smithi100 ceph-mon[39557]: pgmap v1434: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:37.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:37 smithi100 ceph-mon[30138]: pgmap v1434: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:39 smithi179 ceph-mon[33633]: pgmap v1435: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:39.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:39 smithi100 ceph-mon[39557]: pgmap v1435: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:39 smithi100 ceph-mon[30138]: pgmap v1435: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:10:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:10:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:10:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:41 smithi179 ceph-mon[33633]: pgmap v1436: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:41.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:41 smithi100 ceph-mon[30138]: pgmap v1436: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:41 smithi100 ceph-mon[39557]: pgmap v1436: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:43.180 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:10:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:10:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:43 smithi179 ceph-mon[33633]: pgmap v1437: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:43 smithi100 ceph-mon[30138]: pgmap v1437: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:43.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:43 smithi100 ceph-mon[39557]: pgmap v1437: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:44 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:10:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:44 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:10:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:44 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:10:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:44 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:10:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:44 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:10:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:44 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:10:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:44 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:10:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:44 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:10:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:44 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:10:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:45 smithi179 ceph-mon[33633]: pgmap v1438: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:45 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:10:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:45 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:10:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:10:45.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:45 smithi100 ceph-mon[30138]: pgmap v1438: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:45.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:45 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:10:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:45 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:10:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:10:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:45 smithi100 ceph-mon[39557]: pgmap v1438: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:45 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:10:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:45 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:10:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:10:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:47 smithi179 ceph-mon[33633]: pgmap v1439: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:47.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:47 smithi100 ceph-mon[30138]: pgmap v1439: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:47 smithi100 ceph-mon[39557]: pgmap v1439: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:49 smithi179 ceph-mon[33633]: pgmap v1440: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:49 smithi100 ceph-mon[30138]: pgmap v1440: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:49.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:49 smithi100 ceph-mon[39557]: pgmap v1440: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:10:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:10:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:10:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:51 smithi179 ceph-mon[33633]: pgmap v1441: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:51 smithi100 ceph-mon[30138]: pgmap v1441: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:51.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:51 smithi100 ceph-mon[39557]: pgmap v1441: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:53.199 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:10:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:10:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:53 smithi179 ceph-mon[33633]: pgmap v1442: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:53 smithi100 ceph-mon[30138]: pgmap v1442: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:53.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:53 smithi100 ceph-mon[39557]: pgmap v1442: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:10:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:55 smithi179 ceph-mon[33633]: pgmap v1443: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:55 smithi100 ceph-mon[30138]: pgmap v1443: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:55.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:55 smithi100 ceph-mon[39557]: pgmap v1443: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:10:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:57 smithi179 ceph-mon[33633]: pgmap v1444: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:57.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:57 smithi100 ceph-mon[39557]: pgmap v1444: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:57.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:57 smithi100 ceph-mon[30138]: pgmap v1444: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:10:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:10:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:10:59 smithi179 ceph-mon[33633]: pgmap v1445: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:10:59 smithi100 ceph-mon[30138]: pgmap v1445: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:10:59.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:10:59 smithi100 ceph-mon[39557]: pgmap v1445: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:11:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:11:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:11:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:01 smithi179 ceph-mon[33633]: pgmap v1446: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:01 smithi100 ceph-mon[30138]: pgmap v1446: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:01.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:01 smithi100 ceph-mon[39557]: pgmap v1446: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:03.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:11:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:11:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:03 smithi179 ceph-mon[33633]: pgmap v1447: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:03.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:03 smithi100 ceph-mon[39557]: pgmap v1447: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:03 smithi100 ceph-mon[30138]: pgmap v1447: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:05 smithi179 ceph-mon[33633]: pgmap v1448: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:05 smithi100 ceph-mon[39557]: pgmap v1448: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:05 smithi100 ceph-mon[30138]: pgmap v1448: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:06 smithi179 ceph-mon[33633]: pgmap v1449: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:06.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:06 smithi100 ceph-mon[39557]: pgmap v1449: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:06.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:06 smithi100 ceph-mon[30138]: pgmap v1449: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:09 smithi100 ceph-mon[39557]: pgmap v1450: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:09.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:09 smithi100 ceph-mon[30138]: pgmap v1450: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:09 smithi179 ceph-mon[33633]: pgmap v1450: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:11:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:11:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:11:11.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:11 smithi100 ceph-mon[39557]: pgmap v1451: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:11.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:11 smithi100 ceph-mon[30138]: pgmap v1451: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:11 smithi179 ceph-mon[33633]: pgmap v1451: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:13.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:11:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:11:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:13 smithi179 ceph-mon[33633]: pgmap v1452: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:13 smithi100 ceph-mon[39557]: pgmap v1452: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:13.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:13.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:13 smithi100 ceph-mon[30138]: pgmap v1452: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:13.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:14.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:14.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:15.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:15 smithi100 ceph-mon[39557]: pgmap v1453: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:15.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:15 smithi100 ceph-mon[30138]: pgmap v1453: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:15 smithi179 ceph-mon[33633]: pgmap v1453: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:17.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:17 smithi100 ceph-mon[39557]: pgmap v1454: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:17.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:17 smithi100 ceph-mon[30138]: pgmap v1454: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:17 smithi179 ceph-mon[33633]: pgmap v1454: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:19.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:19 smithi100 ceph-mon[39557]: pgmap v1455: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:19.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:19 smithi100 ceph-mon[30138]: pgmap v1455: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:19 smithi179 ceph-mon[33633]: pgmap v1455: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:11:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:11:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:11:21.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:21 smithi100 ceph-mon[39557]: pgmap v1456: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:21.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:21 smithi100 ceph-mon[30138]: pgmap v1456: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:21 smithi179 ceph-mon[33633]: pgmap v1456: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:23 smithi179 ceph-mon[33633]: pgmap v1457: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:23.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:11:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:11:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:23 smithi100 ceph-mon[39557]: pgmap v1457: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:23.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:23 smithi100 ceph-mon[30138]: pgmap v1457: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:24.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:24.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:25 smithi179 ceph-mon[33633]: pgmap v1458: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:25.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:25 smithi100 ceph-mon[39557]: pgmap v1458: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:25.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:25 smithi100 ceph-mon[30138]: pgmap v1458: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:27 smithi179 ceph-mon[33633]: pgmap v1459: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:27.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:27 smithi100 ceph-mon[30138]: pgmap v1459: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:27.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:27 smithi100 ceph-mon[39557]: pgmap v1459: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:29 smithi179 ceph-mon[33633]: pgmap v1460: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:29.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:29 smithi100 ceph-mon[39557]: pgmap v1460: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:29 smithi100 ceph-mon[30138]: pgmap v1460: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:11:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:11:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:11:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:31 smithi179 ceph-mon[33633]: pgmap v1461: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:31.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:31 smithi100 ceph-mon[30138]: pgmap v1461: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:31.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:31 smithi100 ceph-mon[39557]: pgmap v1461: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:33 smithi179 ceph-mon[33633]: pgmap v1462: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:33.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:11:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:11:33.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:33 smithi100 ceph-mon[30138]: pgmap v1462: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:33 smithi100 ceph-mon[39557]: pgmap v1462: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:35 smithi179 ceph-mon[33633]: pgmap v1463: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:35 smithi100 ceph-mon[30138]: pgmap v1463: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:35.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:35 smithi100 ceph-mon[39557]: pgmap v1463: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:37 smithi179 ceph-mon[33633]: pgmap v1464: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:37.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:37 smithi100 ceph-mon[30138]: pgmap v1464: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:37.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:37 smithi100 ceph-mon[39557]: pgmap v1464: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:39 smithi179 ceph-mon[33633]: pgmap v1465: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:39 smithi100 ceph-mon[30138]: pgmap v1465: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:39.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:39 smithi100 ceph-mon[39557]: pgmap v1465: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:11:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:11:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:11:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:41 smithi179 ceph-mon[33633]: pgmap v1466: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:41.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:41 smithi100 ceph-mon[30138]: pgmap v1466: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:41 smithi100 ceph-mon[39557]: pgmap v1466: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:43 smithi179 ceph-mon[33633]: pgmap v1467: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:43.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:11:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:11:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:43 smithi100 ceph-mon[30138]: pgmap v1467: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:43 smithi100 ceph-mon[39557]: pgmap v1467: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:44.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:45 smithi179 ceph-mon[33633]: pgmap v1468: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:45 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:11:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:11:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:45 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:11:45.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:45 smithi100 ceph-mon[39557]: pgmap v1468: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:45.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:45 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:11:45.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:11:45.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:45 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:11:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:45 smithi100 ceph-mon[30138]: pgmap v1468: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:45 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:11:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:11:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:45 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:11:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:46 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:11:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:46 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:11:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:46 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:11:46.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:46 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:11:46.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:46 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:11:46.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:46 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:11:46.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:46 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:11:46.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:46 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:11:46.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:46 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:11:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:47 smithi179 ceph-mon[33633]: pgmap v1469: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:47.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:47 smithi100 ceph-mon[30138]: pgmap v1469: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:47 smithi100 ceph-mon[39557]: pgmap v1469: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:49 smithi179 ceph-mon[33633]: pgmap v1470: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:49 smithi100 ceph-mon[30138]: pgmap v1470: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:49.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:49 smithi100 ceph-mon[39557]: pgmap v1470: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:11:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:11:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:11:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:51 smithi179 ceph-mon[33633]: pgmap v1471: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:51 smithi100 ceph-mon[30138]: pgmap v1471: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:51 smithi100 ceph-mon[39557]: pgmap v1471: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:53.209 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:11:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:11:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:53 smithi179 ceph-mon[33633]: pgmap v1472: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:53 smithi100 ceph-mon[30138]: pgmap v1472: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:53 smithi100 ceph-mon[39557]: pgmap v1472: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:54.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:11:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:55 smithi179 ceph-mon[33633]: pgmap v1473: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:55 smithi100 ceph-mon[30138]: pgmap v1473: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:55 smithi100 ceph-mon[39557]: pgmap v1473: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:11:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:57 smithi179 ceph-mon[33633]: pgmap v1474: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:57.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:57 smithi100 ceph-mon[39557]: pgmap v1474: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:57.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:57 smithi100 ceph-mon[30138]: pgmap v1474: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:11:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:11:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:11:59 smithi179 ceph-mon[33633]: pgmap v1475: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:11:59 smithi100 ceph-mon[30138]: pgmap v1475: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:11:59.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:11:59 smithi100 ceph-mon[39557]: pgmap v1475: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:12:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:12:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:12:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:01 smithi179 ceph-mon[33633]: pgmap v1476: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:01 smithi100 ceph-mon[30138]: pgmap v1476: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:01.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:01 smithi100 ceph-mon[39557]: pgmap v1476: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:03.230 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:12:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:12:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:03 smithi179 ceph-mon[33633]: pgmap v1477: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:03 smithi100 ceph-mon[30138]: pgmap v1477: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:03 smithi100 ceph-mon[39557]: pgmap v1477: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:05 smithi179 ceph-mon[33633]: pgmap v1478: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:05.635 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.635 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.635 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.635 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.635 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.635 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.635 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.635 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.636 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.637 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.638 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.639 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.640 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.641 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.642 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.642 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.642 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.642 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:12:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:05 smithi100 ceph-mon[39557]: pgmap v1478: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:05 smithi100 ceph-mon[30138]: pgmap v1478: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:06 smithi179 ceph-mon[33633]: pgmap v1479: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:06.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:06 smithi100 ceph-mon[39557]: pgmap v1479: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:06.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:06 smithi100 ceph-mon[30138]: pgmap v1479: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:09.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:09 smithi100 ceph-mon[30138]: pgmap v1480: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:09.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:09 smithi100 ceph-mon[39557]: pgmap v1480: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:09 smithi179 ceph-mon[33633]: pgmap v1480: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:12:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:12:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:12:11.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:11 smithi100 ceph-mon[30138]: pgmap v1481: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:11.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:11 smithi100 ceph-mon[39557]: pgmap v1481: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:11 smithi179 ceph-mon[33633]: pgmap v1481: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:13.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:12:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:12:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:13 smithi179 ceph-mon[33633]: pgmap v1482: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:13 smithi100 ceph-mon[30138]: pgmap v1482: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:13 smithi100 ceph-mon[39557]: pgmap v1482: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:14.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:15 smithi179 ceph-mon[33633]: pgmap v1483: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:15 smithi100 ceph-mon[30138]: pgmap v1483: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:15.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:15 smithi100 ceph-mon[39557]: pgmap v1483: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:17.574 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:17 smithi179 ceph-mon[33633]: pgmap v1484: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:17 smithi100 ceph-mon[30138]: pgmap v1484: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:17.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:17 smithi100 ceph-mon[39557]: pgmap v1484: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:19 smithi179 ceph-mon[33633]: pgmap v1485: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:19.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:19 smithi100 ceph-mon[30138]: pgmap v1485: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:19.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:19 smithi100 ceph-mon[39557]: pgmap v1485: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:12:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:12:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:12:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:21 smithi179 ceph-mon[33633]: pgmap v1486: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:21.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:21 smithi100 ceph-mon[30138]: pgmap v1486: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:21.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:21 smithi100 ceph-mon[39557]: pgmap v1486: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:23 smithi179 ceph-mon[33633]: pgmap v1487: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:23.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:12:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:12:23.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:23 smithi100 ceph-mon[30138]: pgmap v1487: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:23.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:23 smithi100 ceph-mon[39557]: pgmap v1487: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:24.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:24.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:24.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:25 smithi179 ceph-mon[33633]: pgmap v1488: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:25.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:25 smithi100 ceph-mon[30138]: pgmap v1488: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:25.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:25 smithi100 ceph-mon[39557]: pgmap v1488: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:27 smithi179 ceph-mon[33633]: pgmap v1489: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:27.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:27 smithi100 ceph-mon[30138]: pgmap v1489: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:27.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:27 smithi100 ceph-mon[39557]: pgmap v1489: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:29 smithi179 ceph-mon[33633]: pgmap v1490: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:29 smithi100 ceph-mon[30138]: pgmap v1490: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:29.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:29 smithi100 ceph-mon[39557]: pgmap v1490: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:12:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:12:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:12:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:31 smithi179 ceph-mon[33633]: pgmap v1491: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:31.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:31 smithi100 ceph-mon[30138]: pgmap v1491: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:31.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:31 smithi100 ceph-mon[39557]: pgmap v1491: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:33 smithi179 ceph-mon[33633]: pgmap v1492: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:33.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:12:32 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:12:33.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:33 smithi100 ceph-mon[30138]: pgmap v1492: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:33 smithi100 ceph-mon[39557]: pgmap v1492: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:35 smithi179 ceph-mon[33633]: pgmap v1493: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:35.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:35 smithi100 ceph-mon[39557]: pgmap v1493: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:35.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:35 smithi100 ceph-mon[30138]: pgmap v1493: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:37 smithi179 ceph-mon[33633]: pgmap v1494: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:37 smithi100 ceph-mon[30138]: pgmap v1494: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:37.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:37 smithi100 ceph-mon[39557]: pgmap v1494: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:39 smithi179 ceph-mon[33633]: pgmap v1495: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:39.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:39 smithi100 ceph-mon[30138]: pgmap v1495: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:39.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:39 smithi100 ceph-mon[39557]: pgmap v1495: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:12:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:12:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:12:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:41 smithi179 ceph-mon[33633]: pgmap v1496: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:41.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:41 smithi100 ceph-mon[39557]: pgmap v1496: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:41 smithi100 ceph-mon[30138]: pgmap v1496: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:43 smithi179 ceph-mon[33633]: pgmap v1497: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:43.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:12:42 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:12:43.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:43 smithi100 ceph-mon[39557]: pgmap v1497: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:43.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:43 smithi100 ceph-mon[30138]: pgmap v1497: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:44.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:45 smithi179 ceph-mon[33633]: pgmap v1498: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:45.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:45 smithi100 ceph-mon[30138]: pgmap v1498: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:45.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:45 smithi100 ceph-mon[39557]: pgmap v1498: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:46 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:12:46.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:46 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:12:46.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:46 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:12:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:47 smithi179 ceph-mon[33633]: pgmap v1499: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:47 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:12:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:47 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:12:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:47 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:12:47.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:47 smithi100 ceph-mon[30138]: pgmap v1499: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:47.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:47 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:12:47.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:47 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:12:47.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:47 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:12:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:47 smithi100 ceph-mon[39557]: pgmap v1499: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:47 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:12:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:47 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:12:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:47 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:12:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:49 smithi179 ceph-mon[33633]: pgmap v1500: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:49.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:49 smithi100 ceph-mon[30138]: pgmap v1500: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:49.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:49 smithi100 ceph-mon[39557]: pgmap v1500: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:12:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:12:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:12:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:51 smithi179 ceph-mon[33633]: pgmap v1501: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:51 smithi100 ceph-mon[30138]: pgmap v1501: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:51.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:51 smithi100 ceph-mon[39557]: pgmap v1501: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:53 smithi179 ceph-mon[33633]: pgmap v1502: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:53.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:12:52 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:12:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:53 smithi100 ceph-mon[30138]: pgmap v1502: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:53.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:53 smithi100 ceph-mon[39557]: pgmap v1502: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:54.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:12:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:55 smithi179 ceph-mon[33633]: pgmap v1503: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:55 smithi100 ceph-mon[30138]: pgmap v1503: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:55.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:55 smithi100 ceph-mon[39557]: pgmap v1503: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:12:57.574 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:57 smithi179 ceph-mon[33633]: pgmap v1504: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:57.654 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:57 smithi100 ceph-mon[39557]: pgmap v1504: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:57.654 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:57 smithi100 ceph-mon[30138]: pgmap v1504: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:12:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:12:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:12:59 smithi179 ceph-mon[33633]: pgmap v1505: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:12:59 smithi100 ceph-mon[30138]: pgmap v1505: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:12:59.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:12:59 smithi100 ceph-mon[39557]: pgmap v1505: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:00 smithi179 ceph-mon[33633]: pgmap v1506: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:00 smithi100 ceph-mon[39557]: pgmap v1506: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:00 smithi100 ceph-mon[30138]: pgmap v1506: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:00.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:13:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:13:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:13:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:03 smithi179 ceph-mon[33633]: pgmap v1507: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:03.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:13:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:13:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:03 smithi100 ceph-mon[30138]: pgmap v1507: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:03 smithi100 ceph-mon[39557]: pgmap v1507: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:05 smithi179 ceph-mon[33633]: pgmap v1508: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:05 smithi100 ceph-mon[30138]: pgmap v1508: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:05 smithi100 ceph-mon[39557]: pgmap v1508: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:07 smithi179 ceph-mon[33633]: pgmap v1509: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:07 smithi100 ceph-mon[30138]: pgmap v1509: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:07.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:07 smithi100 ceph-mon[39557]: pgmap v1509: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:09 smithi179 ceph-mon[33633]: pgmap v1510: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:09.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:09 smithi100 ceph-mon[30138]: pgmap v1510: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:09.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:09 smithi100 ceph-mon[39557]: pgmap v1510: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:13:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:13:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:13:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:11 smithi179 ceph-mon[33633]: pgmap v1511: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:11.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:11 smithi100 ceph-mon[39557]: pgmap v1511: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:11.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:11 smithi100 ceph-mon[30138]: pgmap v1511: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:13.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:13:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:13:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:13 smithi179 ceph-mon[33633]: pgmap v1512: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:13.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:13 smithi100 ceph-mon[39557]: pgmap v1512: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:13.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:13 smithi100 ceph-mon[30138]: pgmap v1512: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:14.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:15 smithi179 ceph-mon[33633]: pgmap v1513: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:15 smithi100 ceph-mon[30138]: pgmap v1513: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:15.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:15 smithi100 ceph-mon[39557]: pgmap v1513: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:17 smithi179 ceph-mon[33633]: pgmap v1514: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:17 smithi100 ceph-mon[30138]: pgmap v1514: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:17 smithi100 ceph-mon[39557]: pgmap v1514: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:19 smithi179 ceph-mon[33633]: pgmap v1515: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:19.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:19 smithi100 ceph-mon[30138]: pgmap v1515: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:19.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:19 smithi100 ceph-mon[39557]: pgmap v1515: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:13:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:13:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:13:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:21 smithi179 ceph-mon[33633]: pgmap v1516: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:21.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:21 smithi100 ceph-mon[39557]: pgmap v1516: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:21.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:21 smithi100 ceph-mon[30138]: pgmap v1516: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:23 smithi179 ceph-mon[33633]: pgmap v1517: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:23.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:13:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:13:23.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:23 smithi100 ceph-mon[39557]: pgmap v1517: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:23.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:23 smithi100 ceph-mon[30138]: pgmap v1517: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:24.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:24.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:25 smithi179 ceph-mon[33633]: pgmap v1518: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:25.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:25 smithi100 ceph-mon[30138]: pgmap v1518: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:25.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:25 smithi100 ceph-mon[39557]: pgmap v1518: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:27 smithi179 ceph-mon[33633]: pgmap v1519: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:27.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:27 smithi100 ceph-mon[30138]: pgmap v1519: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:27.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:27 smithi100 ceph-mon[39557]: pgmap v1519: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:29 smithi179 ceph-mon[33633]: pgmap v1520: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:29 smithi100 ceph-mon[30138]: pgmap v1520: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:29.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:29 smithi100 ceph-mon[39557]: pgmap v1520: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:13:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:13:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:13:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:31 smithi179 ceph-mon[33633]: pgmap v1521: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:31.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:31 smithi100 ceph-mon[39557]: pgmap v1521: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:31.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:31 smithi100 ceph-mon[30138]: pgmap v1521: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:33 smithi179 ceph-mon[33633]: pgmap v1522: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:33.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:13:33 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:13:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:33 smithi100 ceph-mon[39557]: pgmap v1522: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:33 smithi100 ceph-mon[30138]: pgmap v1522: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:35 smithi179 ceph-mon[33633]: pgmap v1523: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:35 smithi100 ceph-mon[30138]: pgmap v1523: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:35.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:35 smithi100 ceph-mon[39557]: pgmap v1523: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:37 smithi179 ceph-mon[33633]: pgmap v1524: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:37.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:37 smithi100 ceph-mon[30138]: pgmap v1524: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:37 smithi100 ceph-mon[39557]: pgmap v1524: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:39 smithi179 ceph-mon[33633]: pgmap v1525: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:39 smithi100 ceph-mon[30138]: pgmap v1525: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:39.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:39 smithi100 ceph-mon[39557]: pgmap v1525: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:13:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:13:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:13:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:41 smithi179 ceph-mon[33633]: pgmap v1526: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:41.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:41 smithi100 ceph-mon[39557]: pgmap v1526: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:41 smithi100 ceph-mon[30138]: pgmap v1526: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:43.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:13:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:13:43.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:43 smithi179 ceph-mon[33633]: pgmap v1527: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:43.322 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:43 smithi100 ceph-mon[30138]: pgmap v1527: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:43 smithi100 ceph-mon[39557]: pgmap v1527: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:44.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:45 smithi179 ceph-mon[33633]: pgmap v1528: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:45.647 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.647 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.647 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.647 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.647 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.648 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.648 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.648 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.648 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.648 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.648 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.648 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.648 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.649 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.650 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.651 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.652 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.653 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.654 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:13:45.655 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:45 smithi100 ceph-mon[30138]: pgmap v1528: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:45.655 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:45 smithi100 ceph-mon[39557]: pgmap v1528: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:46 smithi179 ceph-mon[33633]: pgmap v1529: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:46.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:46 smithi100 ceph-mon[39557]: pgmap v1529: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:46.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:46 smithi100 ceph-mon[30138]: pgmap v1529: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:47 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:13:47.590 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:47 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:13:47.590 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:47 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:13:48.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:48 smithi179 ceph-mon[33633]: pgmap v1530: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:48.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:48 smithi100 ceph-mon[39557]: pgmap v1530: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:48.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:48 smithi100 ceph-mon[30138]: pgmap v1530: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:50.649 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:13:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:13:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:13:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:51 smithi179 ceph-mon[33633]: pgmap v1531: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:51 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:13:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:51 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:13:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:51 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:13:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:51 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:13:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:51 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:13:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[39557]: pgmap v1531: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:13:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:13:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:13:51.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:13:51.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:13:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[30138]: pgmap v1531: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:51.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:13:51.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:13:51.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:13:51.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:13:51.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:51 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:13:53.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:53 smithi179 ceph-mon[33633]: pgmap v1532: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:53.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:13:53 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:13:53.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:53 smithi100 ceph-mon[39557]: pgmap v1532: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:53 smithi100 ceph-mon[30138]: pgmap v1532: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:13:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:55 smithi179 ceph-mon[33633]: pgmap v1533: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:55 smithi100 ceph-mon[30138]: pgmap v1533: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:55.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:55 smithi100 ceph-mon[39557]: pgmap v1533: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:13:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:57 smithi179 ceph-mon[33633]: pgmap v1534: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:57.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:57 smithi100 ceph-mon[30138]: pgmap v1534: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:57.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:57 smithi100 ceph-mon[39557]: pgmap v1534: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:13:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:13:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:13:59 smithi179 ceph-mon[33633]: pgmap v1535: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:13:59 smithi100 ceph-mon[30138]: pgmap v1535: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:13:59.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:13:59 smithi100 ceph-mon[39557]: pgmap v1535: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:14:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:14:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:14:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:01 smithi179 ceph-mon[33633]: pgmap v1536: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:01 smithi100 ceph-mon[30138]: pgmap v1536: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:01 smithi100 ceph-mon[39557]: pgmap v1536: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:03.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:14:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:14:03.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:03 smithi179 ceph-mon[33633]: pgmap v1537: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:03 smithi100 ceph-mon[30138]: pgmap v1537: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:03 smithi100 ceph-mon[39557]: pgmap v1537: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:05 smithi179 ceph-mon[33633]: pgmap v1538: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:05 smithi100 ceph-mon[30138]: pgmap v1538: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:05 smithi100 ceph-mon[39557]: pgmap v1538: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:07 smithi179 ceph-mon[33633]: pgmap v1539: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:07 smithi100 ceph-mon[30138]: pgmap v1539: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:07 smithi100 ceph-mon[39557]: pgmap v1539: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:09 smithi179 ceph-mon[33633]: pgmap v1540: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:09.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:09 smithi100 ceph-mon[30138]: pgmap v1540: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:09.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:09 smithi100 ceph-mon[39557]: pgmap v1540: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:14:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:14:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:14:11.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:11 smithi100 ceph-mon[30138]: pgmap v1541: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:11.566 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:11 smithi100 ceph-mon[39557]: pgmap v1541: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:11 smithi179 ceph-mon[33633]: pgmap v1541: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:13.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:14:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:14:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:13 smithi179 ceph-mon[33633]: pgmap v1542: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:13.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:13.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:13 smithi100 ceph-mon[39557]: pgmap v1542: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:13 smithi100 ceph-mon[30138]: pgmap v1542: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:15 smithi179 ceph-mon[33633]: pgmap v1543: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:15 smithi100 ceph-mon[30138]: pgmap v1543: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:15 smithi100 ceph-mon[39557]: pgmap v1543: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:17 smithi179 ceph-mon[33633]: pgmap v1544: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:17 smithi100 ceph-mon[30138]: pgmap v1544: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:17 smithi100 ceph-mon[39557]: pgmap v1544: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:19 smithi179 ceph-mon[33633]: pgmap v1545: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:19.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:19 smithi100 ceph-mon[30138]: pgmap v1545: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:19.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:19 smithi100 ceph-mon[39557]: pgmap v1545: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:14:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:14:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:14:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:21 smithi179 ceph-mon[33633]: pgmap v1546: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:21.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:21 smithi100 ceph-mon[30138]: pgmap v1546: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:21.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:21 smithi100 ceph-mon[39557]: pgmap v1546: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:23.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:14:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:14:23.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:23 smithi179 ceph-mon[33633]: pgmap v1547: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:23.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:23 smithi100 ceph-mon[30138]: pgmap v1547: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:23.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:23 smithi100 ceph-mon[39557]: pgmap v1547: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:24.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:24.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:25 smithi179 ceph-mon[33633]: pgmap v1548: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:25.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:25 smithi100 ceph-mon[39557]: pgmap v1548: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:25.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:25 smithi100 ceph-mon[30138]: pgmap v1548: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:27 smithi179 ceph-mon[33633]: pgmap v1549: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:27.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:27 smithi100 ceph-mon[30138]: pgmap v1549: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:27.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:27 smithi100 ceph-mon[39557]: pgmap v1549: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:29 smithi179 ceph-mon[33633]: pgmap v1550: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:29.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:29 smithi100 ceph-mon[39557]: pgmap v1550: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:29 smithi100 ceph-mon[30138]: pgmap v1550: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:30 smithi179 ceph-mon[33633]: pgmap v1551: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:30.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:30 smithi100 ceph-mon[30138]: pgmap v1551: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:30 smithi100 ceph-mon[39557]: pgmap v1551: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:30.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:14:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:14:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:14:33.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:33 smithi179 ceph-mon[33633]: pgmap v1552: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:33.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:14:33 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:14:33.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:33 smithi100 ceph-mon[30138]: pgmap v1552: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:33.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:33 smithi100 ceph-mon[39557]: pgmap v1552: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:34.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:34.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:35 smithi179 ceph-mon[33633]: pgmap v1553: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:35 smithi100 ceph-mon[30138]: pgmap v1553: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:35 smithi100 ceph-mon[39557]: pgmap v1553: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:37 smithi179 ceph-mon[33633]: pgmap v1554: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:37.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:37 smithi100 ceph-mon[30138]: pgmap v1554: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:37 smithi100 ceph-mon[39557]: pgmap v1554: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:39 smithi179 ceph-mon[33633]: pgmap v1555: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:39.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:39 smithi100 ceph-mon[39557]: pgmap v1555: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:39 smithi100 ceph-mon[30138]: pgmap v1555: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:14:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:14:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:14:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:41 smithi179 ceph-mon[33633]: pgmap v1556: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:41.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:41 smithi100 ceph-mon[39557]: pgmap v1556: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:41 smithi100 ceph-mon[30138]: pgmap v1556: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:43 smithi179 ceph-mon[33633]: pgmap v1557: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:43.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:14:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:14:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:43 smithi100 ceph-mon[30138]: pgmap v1557: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:43 smithi100 ceph-mon[39557]: pgmap v1557: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:44.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:45 smithi179 ceph-mon[33633]: pgmap v1558: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:45.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:45 smithi100 ceph-mon[39557]: pgmap v1558: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:45 smithi100 ceph-mon[30138]: pgmap v1558: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:47 smithi179 ceph-mon[33633]: pgmap v1559: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:47 smithi100 ceph-mon[39557]: pgmap v1559: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:47.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:47 smithi100 ceph-mon[30138]: pgmap v1559: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:49 smithi179 ceph-mon[33633]: pgmap v1560: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:49.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:49 smithi100 ceph-mon[39557]: pgmap v1560: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:49.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:49 smithi100 ceph-mon[30138]: pgmap v1560: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:14:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:14:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:14:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:51 smithi179 ceph-mon[33633]: pgmap v1561: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:51 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:14:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:51 smithi100 ceph-mon[30138]: pgmap v1561: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:51 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:14:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:51 smithi100 ceph-mon[39557]: pgmap v1561: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:51 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:14:53.404 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:53 smithi179 ceph-mon[33633]: pgmap v1562: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:53.404 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:14:53 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:14:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:53 smithi100 ceph-mon[30138]: pgmap v1562: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:53.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:53 smithi100 ceph-mon[39557]: pgmap v1562: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:54.552 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:14:55.499 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:55 smithi179 ceph-mon[33633]: pgmap v1563: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:55.499 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:55 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:14:55.499 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:55 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:14:55.499 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:55 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:14:55.499 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:55 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:14:55.499 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:55 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:14:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[30138]: pgmap v1563: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:14:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:14:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:14:55.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:14:55.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:14:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[39557]: pgmap v1563: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:14:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:14:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:14:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:14:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:14:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:55 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:14:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:57 smithi179 ceph-mon[33633]: pgmap v1564: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:57.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:57 smithi100 ceph-mon[30138]: pgmap v1564: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:57.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:57 smithi100 ceph-mon[39557]: pgmap v1564: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:14:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:14:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:14:59 smithi179 ceph-mon[33633]: pgmap v1565: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:14:59 smithi100 ceph-mon[30138]: pgmap v1565: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:14:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:14:59 smithi100 ceph-mon[39557]: pgmap v1565: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:15:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:15:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:15:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:01 smithi179 ceph-mon[33633]: pgmap v1566: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:01 smithi100 ceph-mon[30138]: pgmap v1566: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:01 smithi100 ceph-mon[39557]: pgmap v1566: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:03 smithi179 ceph-mon[33633]: pgmap v1567: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:03.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:15:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:15:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:03 smithi100 ceph-mon[30138]: pgmap v1567: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:03.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:03 smithi100 ceph-mon[39557]: pgmap v1567: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:05.559 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:05 smithi179 ceph-mon[33633]: pgmap v1568: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:05 smithi100 ceph-mon[39557]: pgmap v1568: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:05 smithi100 ceph-mon[30138]: pgmap v1568: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:06.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:06 smithi179 ceph-mon[33633]: pgmap v1569: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:06.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:06 smithi100 ceph-mon[30138]: pgmap v1569: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:06.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:06 smithi100 ceph-mon[39557]: pgmap v1569: 260 pgs: 260 active+clean; 582 KiB data, 864 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:09 smithi179 ceph-mon[33633]: pgmap v1570: 260 pgs: 260 active+clean; 582 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-14T18:15:09.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:09 smithi100 ceph-mon[30138]: pgmap v1570: 260 pgs: 260 active+clean; 582 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-14T18:15:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:09 smithi100 ceph-mon[39557]: pgmap v1570: 260 pgs: 260 active+clean; 582 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-14T18:15:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:15:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:15:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:15:11.566 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:11 smithi100 ceph-mon[30138]: pgmap v1571: 260 pgs: 260 active+clean; 582 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-14T18:15:11.566 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:11 smithi100 ceph-mon[39557]: pgmap v1571: 260 pgs: 260 active+clean; 582 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-14T18:15:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:11 smithi179 ceph-mon[33633]: pgmap v1571: 260 pgs: 260 active+clean; 582 KiB data, 881 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-14T18:15:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:13 smithi179 ceph-mon[33633]: pgmap v1572: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:13.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:15:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:15:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:13 smithi100 ceph-mon[30138]: pgmap v1572: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:13 smithi100 ceph-mon[39557]: pgmap v1572: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:14.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:15 smithi179 ceph-mon[33633]: pgmap v1573: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:15.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:15 smithi100 ceph-mon[39557]: pgmap v1573: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:15 smithi100 ceph-mon[30138]: pgmap v1573: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:17 smithi179 ceph-mon[33633]: pgmap v1574: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:17 smithi100 ceph-mon[30138]: pgmap v1574: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:17.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:17 smithi100 ceph-mon[39557]: pgmap v1574: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:19 smithi179 ceph-mon[33633]: pgmap v1575: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:19.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:19 smithi100 ceph-mon[39557]: pgmap v1575: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:19.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:19 smithi100 ceph-mon[30138]: pgmap v1575: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-14T18:15:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:15:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:15:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:15:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:21 smithi179 ceph-mon[33633]: pgmap v1576: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-02-14T18:15:21.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:21 smithi100 ceph-mon[30138]: pgmap v1576: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-02-14T18:15:21.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:21 smithi100 ceph-mon[39557]: pgmap v1576: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-02-14T18:15:23.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:15:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:15:23.374 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:23 smithi179 ceph-mon[33633]: pgmap v1577: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-02-14T18:15:23.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:23 smithi100 ceph-mon[30138]: pgmap v1577: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-02-14T18:15:23.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:23 smithi100 ceph-mon[39557]: pgmap v1577: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-02-14T18:15:24.568 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:24.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:24.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:24.656 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.657 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.658 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.659 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.660 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.661 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.662 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:24.663 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:15:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:25 smithi179 ceph-mon[33633]: pgmap v1578: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:25.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:25 smithi100 ceph-mon[30138]: pgmap v1578: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:25 smithi100 ceph-mon[39557]: pgmap v1578: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:27 smithi179 ceph-mon[33633]: pgmap v1579: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:15:27.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:27 smithi100 ceph-mon[30138]: pgmap v1579: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:15:27.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:27 smithi100 ceph-mon[39557]: pgmap v1579: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:15:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:28.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:29 smithi179 ceph-mon[33633]: pgmap v1580: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:29.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:29 smithi100 ceph-mon[39557]: pgmap v1580: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:29 smithi100 ceph-mon[30138]: pgmap v1580: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:15:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:15:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:15:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:31 smithi179 ceph-mon[33633]: pgmap v1581: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:31.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:31 smithi100 ceph-mon[39557]: pgmap v1581: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:31.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:31 smithi100 ceph-mon[30138]: pgmap v1581: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:33 smithi179 ceph-mon[33633]: pgmap v1582: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:33.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:15:33 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:15:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:33 smithi100 ceph-mon[39557]: pgmap v1582: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:33 smithi100 ceph-mon[30138]: pgmap v1582: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:35 smithi179 ceph-mon[33633]: pgmap v1583: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:35 smithi100 ceph-mon[30138]: pgmap v1583: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:35.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:35 smithi100 ceph-mon[39557]: pgmap v1583: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:37 smithi179 ceph-mon[33633]: pgmap v1584: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:37.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:37 smithi100 ceph-mon[30138]: pgmap v1584: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:37 smithi100 ceph-mon[39557]: pgmap v1584: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:38 smithi179 ceph-mon[33633]: pgmap v1585: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:38.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:38 smithi100 ceph-mon[30138]: pgmap v1585: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:38.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:38 smithi100 ceph-mon[39557]: pgmap v1585: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:15:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:15:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:15:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:41 smithi179 ceph-mon[33633]: pgmap v1586: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:41.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:41 smithi100 ceph-mon[30138]: pgmap v1586: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:41 smithi100 ceph-mon[39557]: pgmap v1586: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:43 smithi179 ceph-mon[33633]: pgmap v1587: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:43.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:15:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:15:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:43 smithi100 ceph-mon[30138]: pgmap v1587: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:43 smithi100 ceph-mon[39557]: pgmap v1587: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:44.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:44.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:45 smithi179 ceph-mon[33633]: pgmap v1588: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:15:45.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:45 smithi100 ceph-mon[39557]: pgmap v1588: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:15:45.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:45 smithi100 ceph-mon[30138]: pgmap v1588: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:15:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:47 smithi179 ceph-mon[33633]: pgmap v1589: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:15:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:47 smithi100 ceph-mon[39557]: pgmap v1589: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:15:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:47 smithi100 ceph-mon[30138]: pgmap v1589: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:15:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:49 smithi179 ceph-mon[33633]: pgmap v1590: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:49.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:49 smithi100 ceph-mon[39557]: pgmap v1590: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:49 smithi100 ceph-mon[30138]: pgmap v1590: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:15:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:15:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:15:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:51 smithi179 ceph-mon[33633]: pgmap v1591: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:15:51.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:51 smithi100 ceph-mon[39557]: pgmap v1591: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:15:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:51 smithi100 ceph-mon[30138]: pgmap v1591: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:15:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:53 smithi179 ceph-mon[33633]: pgmap v1592: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:53.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:15:53 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:15:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:53 smithi100 ceph-mon[30138]: pgmap v1592: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:53.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:53 smithi100 ceph-mon[39557]: pgmap v1592: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:15:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:55 smithi179 ceph-mon[33633]: pgmap v1593: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:55 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:15:55.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:55 smithi100 ceph-mon[39557]: pgmap v1593: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:55.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:55 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:15:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:55 smithi100 ceph-mon[30138]: pgmap v1593: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:15:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:55 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:15:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:57 smithi179 ceph-mon[33633]: pgmap v1594: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:15:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:57 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:15:57.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:57 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:15:57.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:57 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:15:57.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:57 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:15:57.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:57 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:15:57.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[30138]: pgmap v1594: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[39557]: pgmap v1594: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:15:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:57 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:15:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:15:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:15:59 smithi179 ceph-mon[33633]: pgmap v1595: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:59.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:15:59 smithi100 ceph-mon[39557]: pgmap v1595: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:15:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:15:59 smithi100 ceph-mon[30138]: pgmap v1595: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:16:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:16:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:16:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:01 smithi179 ceph-mon[33633]: pgmap v1596: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:01 smithi100 ceph-mon[30138]: pgmap v1596: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:01.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:01 smithi100 ceph-mon[39557]: pgmap v1596: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:03 smithi179 ceph-mon[33633]: pgmap v1597: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:03.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:16:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:16:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:03 smithi100 ceph-mon[30138]: pgmap v1597: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:03 smithi100 ceph-mon[39557]: pgmap v1597: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:05 smithi179 ceph-mon[33633]: pgmap v1598: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:05 smithi100 ceph-mon[30138]: pgmap v1598: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:05.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:05 smithi100 ceph-mon[39557]: pgmap v1598: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:07 smithi179 ceph-mon[33633]: pgmap v1599: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:07 smithi100 ceph-mon[30138]: pgmap v1599: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:07.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:07 smithi100 ceph-mon[39557]: pgmap v1599: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:09 smithi179 ceph-mon[33633]: pgmap v1600: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:09.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:09 smithi100 ceph-mon[30138]: pgmap v1600: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:09.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:09 smithi100 ceph-mon[39557]: pgmap v1600: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:16:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:16:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:16:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:11 smithi179 ceph-mon[33633]: pgmap v1601: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:11.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:11 smithi100 ceph-mon[30138]: pgmap v1601: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:11.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:11 smithi100 ceph-mon[39557]: pgmap v1601: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:13 smithi179 ceph-mon[33633]: pgmap v1602: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:13.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:16:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:16:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:13 smithi100 ceph-mon[30138]: pgmap v1602: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:13 smithi100 ceph-mon[39557]: pgmap v1602: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:14.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:15 smithi179 ceph-mon[33633]: pgmap v1603: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:15 smithi100 ceph-mon[30138]: pgmap v1603: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:15 smithi100 ceph-mon[39557]: pgmap v1603: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:17 smithi179 ceph-mon[33633]: pgmap v1604: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:17 smithi100 ceph-mon[30138]: pgmap v1604: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:17.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:17 smithi100 ceph-mon[39557]: pgmap v1604: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:19 smithi179 ceph-mon[33633]: pgmap v1605: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:19.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:19 smithi100 ceph-mon[30138]: pgmap v1605: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:19.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:19 smithi100 ceph-mon[39557]: pgmap v1605: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:16:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:16:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:16:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:21 smithi179 ceph-mon[33633]: pgmap v1606: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:21.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:21 smithi100 ceph-mon[30138]: pgmap v1606: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:21.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:21 smithi100 ceph-mon[39557]: pgmap v1606: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:23 smithi179 ceph-mon[33633]: pgmap v1607: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:23.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:16:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:16:23.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:23 smithi100 ceph-mon[39557]: pgmap v1607: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:23.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:23 smithi100 ceph-mon[30138]: pgmap v1607: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:25 smithi179 ceph-mon[33633]: pgmap v1608: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:25.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:25 smithi100 ceph-mon[30138]: pgmap v1608: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:25.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:25 smithi100 ceph-mon[39557]: pgmap v1608: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:27 smithi179 ceph-mon[33633]: pgmap v1609: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:27.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:27 smithi100 ceph-mon[39557]: pgmap v1609: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:27.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:27 smithi100 ceph-mon[30138]: pgmap v1609: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:28 smithi179 ceph-mon[33633]: pgmap v1610: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:28 smithi100 ceph-mon[39557]: pgmap v1610: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:28 smithi100 ceph-mon[30138]: pgmap v1610: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:16:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:16:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:16:31.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:31 smithi179 ceph-mon[33633]: pgmap v1611: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:31.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:31 smithi100 ceph-mon[39557]: pgmap v1611: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:31.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:31 smithi100 ceph-mon[30138]: pgmap v1611: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:33 smithi179 ceph-mon[33633]: pgmap v1612: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:33.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:16:33 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:16:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:33 smithi100 ceph-mon[39557]: pgmap v1612: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:33.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:33 smithi100 ceph-mon[30138]: pgmap v1612: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:35 smithi179 ceph-mon[33633]: pgmap v1613: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:35 smithi100 ceph-mon[30138]: pgmap v1613: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:35 smithi100 ceph-mon[39557]: pgmap v1613: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:37 smithi179 ceph-mon[33633]: pgmap v1614: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:37.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:37 smithi100 ceph-mon[30138]: pgmap v1614: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:37.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:37 smithi100 ceph-mon[39557]: pgmap v1614: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:39 smithi179 ceph-mon[33633]: pgmap v1615: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:39.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:39 smithi100 ceph-mon[39557]: pgmap v1615: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:39 smithi100 ceph-mon[30138]: pgmap v1615: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:16:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:16:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:16:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:41 smithi179 ceph-mon[33633]: pgmap v1616: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:41.571 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:16:41 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a[66334]: logger=cleanup t=2024-02-14T18:16:41.461408042Z level=info msg="Completed cleanup jobs" duration=5.683053ms 2024-02-14T18:16:41.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:41 smithi100 ceph-mon[30138]: pgmap v1616: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:41.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:41 smithi100 ceph-mon[39557]: pgmap v1616: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:43.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:16:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:16:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:43 smithi179 ceph-mon[33633]: pgmap v1617: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:43.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:43 smithi100 ceph-mon[39557]: pgmap v1617: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:43 smithi100 ceph-mon[30138]: pgmap v1617: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:44.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:45 smithi179 ceph-mon[33633]: pgmap v1618: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:45.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:45 smithi100 ceph-mon[30138]: pgmap v1618: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:45 smithi100 ceph-mon[39557]: pgmap v1618: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:47 smithi179 ceph-mon[33633]: pgmap v1619: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:47.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:47 smithi100 ceph-mon[30138]: pgmap v1619: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:47.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:47 smithi100 ceph-mon[39557]: pgmap v1619: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:49 smithi179 ceph-mon[33633]: pgmap v1620: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:49 smithi100 ceph-mon[30138]: pgmap v1620: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:49.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:49 smithi100 ceph-mon[39557]: pgmap v1620: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:16:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:16:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:16:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:51 smithi179 ceph-mon[33633]: pgmap v1621: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:51 smithi100 ceph-mon[30138]: pgmap v1621: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:51 smithi100 ceph-mon[39557]: pgmap v1621: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:52.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:52 smithi100 ceph-mon[30138]: pgmap v1622: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:52 smithi100 ceph-mon[39557]: pgmap v1622: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:52.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:52 smithi179 ceph-mon[33633]: pgmap v1622: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:53 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:53.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:16:53 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:16:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:53 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:53 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:16:54.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:54 smithi100 ceph-mon[30138]: pgmap v1623: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:54 smithi100 ceph-mon[39557]: pgmap v1623: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:54.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:54 smithi179 ceph-mon[33633]: pgmap v1623: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:16:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:57 smithi179 ceph-mon[33633]: pgmap v1624: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:57 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:16:57.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:57 smithi100 ceph-mon[39557]: pgmap v1624: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:57.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:57 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:16:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:57 smithi100 ceph-mon[30138]: pgmap v1624: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:16:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:57 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:16:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:58 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:16:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:58 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:16:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:16:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:16:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:58 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:16:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:16:58 smithi179 ceph-mon[33633]: pgmap v1625: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:58.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:16:58.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:16:58.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[30138]: pgmap v1625: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:16:58.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:16:58 smithi100 ceph-mon[39557]: pgmap v1625: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:17:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:17:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:17:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:01 smithi179 ceph-mon[33633]: pgmap v1626: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:01 smithi100 ceph-mon[30138]: pgmap v1626: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:01 smithi100 ceph-mon[39557]: pgmap v1626: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:03.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:17:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:17:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:03 smithi179 ceph-mon[33633]: pgmap v1627: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:03 smithi100 ceph-mon[30138]: pgmap v1627: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:03 smithi100 ceph-mon[39557]: pgmap v1627: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:04.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.668 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.669 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.670 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.671 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.672 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.673 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.674 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:04.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:05 smithi179 ceph-mon[33633]: pgmap v1628: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:05 smithi100 ceph-mon[30138]: pgmap v1628: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:05.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:05 smithi100 ceph-mon[39557]: pgmap v1628: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:07 smithi179 ceph-mon[33633]: pgmap v1629: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:07 smithi100 ceph-mon[30138]: pgmap v1629: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:07.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:07 smithi100 ceph-mon[39557]: pgmap v1629: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:09 smithi179 ceph-mon[33633]: pgmap v1630: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:09.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:09 smithi100 ceph-mon[39557]: pgmap v1630: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:09 smithi100 ceph-mon[30138]: pgmap v1630: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:17:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:17:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:17:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:11 smithi179 ceph-mon[33633]: pgmap v1631: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:11.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:11 smithi100 ceph-mon[30138]: pgmap v1631: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:11 smithi100 ceph-mon[39557]: pgmap v1631: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:13 smithi179 ceph-mon[33633]: pgmap v1632: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:13.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:17:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:17:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:13 smithi100 ceph-mon[30138]: pgmap v1632: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:13 smithi100 ceph-mon[39557]: pgmap v1632: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:15 smithi179 ceph-mon[33633]: pgmap v1633: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:15 smithi100 ceph-mon[30138]: pgmap v1633: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:15.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:15 smithi100 ceph-mon[39557]: pgmap v1633: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:16.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:16 smithi179 ceph-mon[33633]: pgmap v1634: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:16.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:16 smithi100 ceph-mon[39557]: pgmap v1634: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:16.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:16 smithi100 ceph-mon[30138]: pgmap v1634: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:19 smithi179 ceph-mon[33633]: pgmap v1635: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:19.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:19 smithi100 ceph-mon[30138]: pgmap v1635: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:19.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:19 smithi100 ceph-mon[39557]: pgmap v1635: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:20.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:17:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:17:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:17:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:21 smithi179 ceph-mon[33633]: pgmap v1636: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:21.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:21 smithi100 ceph-mon[30138]: pgmap v1636: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:21.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:21 smithi100 ceph-mon[39557]: pgmap v1636: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:23.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:23 smithi179 ceph-mon[33633]: pgmap v1637: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:23.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:17:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:17:23.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:23 smithi100 ceph-mon[30138]: pgmap v1637: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:23.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:23 smithi100 ceph-mon[39557]: pgmap v1637: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:24.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:24.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:24.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:25.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:25 smithi179 ceph-mon[33633]: pgmap v1638: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:25.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:25 smithi100 ceph-mon[39557]: pgmap v1638: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:25.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:25 smithi100 ceph-mon[30138]: pgmap v1638: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:27.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:27 smithi179 ceph-mon[33633]: pgmap v1639: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:27.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:27 smithi100 ceph-mon[30138]: pgmap v1639: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:27.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:27 smithi100 ceph-mon[39557]: pgmap v1639: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:29 smithi179 ceph-mon[33633]: pgmap v1640: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:29.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:29 smithi100 ceph-mon[39557]: pgmap v1640: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:29 smithi100 ceph-mon[30138]: pgmap v1640: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:30.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:30 smithi179 ceph-mon[33633]: pgmap v1641: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:30.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:30 smithi100 ceph-mon[30138]: pgmap v1641: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:30 smithi100 ceph-mon[39557]: pgmap v1641: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:30.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:17:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:17:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:17:33.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:17:33 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:17:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:33 smithi179 ceph-mon[33633]: pgmap v1642: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:33.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:33 smithi100 ceph-mon[30138]: pgmap v1642: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:33.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:33 smithi100 ceph-mon[39557]: pgmap v1642: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:35 smithi179 ceph-mon[33633]: pgmap v1643: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:35 smithi100 ceph-mon[30138]: pgmap v1643: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:35 smithi100 ceph-mon[39557]: pgmap v1643: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:37 smithi179 ceph-mon[33633]: pgmap v1644: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:37 smithi100 ceph-mon[39557]: pgmap v1644: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:37 smithi100 ceph-mon[30138]: pgmap v1644: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:39 smithi179 ceph-mon[33633]: pgmap v1645: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:39 smithi100 ceph-mon[30138]: pgmap v1645: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:39.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:39 smithi100 ceph-mon[39557]: pgmap v1645: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:17:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:17:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:17:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:41 smithi179 ceph-mon[33633]: pgmap v1646: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:41.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:41 smithi100 ceph-mon[30138]: pgmap v1646: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:41 smithi100 ceph-mon[39557]: pgmap v1646: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:43.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:17:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:17:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:43 smithi179 ceph-mon[33633]: pgmap v1647: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:43.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:43.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:43 smithi100 ceph-mon[30138]: pgmap v1647: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:43 smithi100 ceph-mon[39557]: pgmap v1647: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:44.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:44.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:45.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:45 smithi179 ceph-mon[33633]: pgmap v1648: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:45.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:45 smithi100 ceph-mon[39557]: pgmap v1648: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:45.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:45 smithi100 ceph-mon[30138]: pgmap v1648: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:46.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:46 smithi179 ceph-mon[33633]: pgmap v1649: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:46.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:46 smithi100 ceph-mon[39557]: pgmap v1649: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:46.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:46 smithi100 ceph-mon[30138]: pgmap v1649: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:49 smithi179 ceph-mon[33633]: pgmap v1650: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:49 smithi100 ceph-mon[30138]: pgmap v1650: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:49.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:49 smithi100 ceph-mon[39557]: pgmap v1650: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:17:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:17:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:17:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:51 smithi179 ceph-mon[33633]: pgmap v1651: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:51 smithi100 ceph-mon[30138]: pgmap v1651: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:51.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:51 smithi100 ceph-mon[39557]: pgmap v1651: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:53.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:53 smithi179 ceph-mon[33633]: pgmap v1652: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:53.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:17:53 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:17:53.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:53 smithi100 ceph-mon[30138]: pgmap v1652: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:53.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:53 smithi100 ceph-mon[39557]: pgmap v1652: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:17:54.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:54.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:17:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:55 smithi179 ceph-mon[33633]: pgmap v1653: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:55 smithi100 ceph-mon[30138]: pgmap v1653: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:55.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:55 smithi100 ceph-mon[39557]: pgmap v1653: 260 pgs: 260 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:56 smithi179 ceph-mon[33633]: osdmap e755: 8 total, 8 up, 8 in 2024-02-14T18:17:56.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:56 smithi179 ceph-mon[33633]: pgmap v1655: 228 pgs: 228 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:17:56.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:56 smithi100 ceph-mon[30138]: osdmap e755: 8 total, 8 up, 8 in 2024-02-14T18:17:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:56 smithi100 ceph-mon[30138]: pgmap v1655: 228 pgs: 228 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:17:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:56 smithi100 ceph-mon[39557]: osdmap e755: 8 total, 8 up, 8 in 2024-02-14T18:17:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:56 smithi100 ceph-mon[39557]: pgmap v1655: 228 pgs: 228 active+clean; 582 KiB data, 885 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:17:57.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:57 smithi100 ceph-mon[39557]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:17:57.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:57 smithi100 ceph-mon[39557]: osdmap e756: 8 total, 8 up, 8 in 2024-02-14T18:17:57.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:57 smithi100 ceph-mon[30138]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:17:57.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:57 smithi100 ceph-mon[30138]: osdmap e756: 8 total, 8 up, 8 in 2024-02-14T18:17:57.675 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.676 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.677 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.678 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 1 sec 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: started 512 aios 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4421600 2024-02-14T18:17:57.679 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47669e0 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4767380 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4766e90 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4767810 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4767cb0 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47687a0 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4768560 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4768c10 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4768670 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47691f0 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4768d50 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4768e60 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4768f70 2024-02-14T18:17:57.680 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4769320 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4769470 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4769a20 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4769580 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4769690 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47697a0 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47698d0 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4769fd0 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4769b30 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4769c40 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4769d50 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4769ec0 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476a5c0 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476a120 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476a250 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476a360 2024-02-14T18:17:57.681 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476a490 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476ab70 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476a6d0 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476a7e0 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476a930 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476aa60 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476b160 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476acc0 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476add0 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476aee0 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476aff0 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476b750 2024-02-14T18:17:57.682 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476b2b0 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476b3e0 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476b4f0 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476b860 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476b970 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476ba80 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476bb90 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476c1e0 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476bd40 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476be50 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476bf60 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476c070 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476c790 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476c2f0 2024-02-14T18:17:57.683 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476c400 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476c510 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476c660 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476cd40 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476c8a0 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476c9b0 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476cac0 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476cbd0 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476d330 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476ce50 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476cfb0 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476d0c0 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476d440 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476d570 2024-02-14T18:17:57.684 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476db40 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476d6a0 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476d7b0 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476d8c0 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476d9f0 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476e0f0 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476dc50 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476dda0 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476deb0 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476dfc0 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476e6a0 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476e200 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476e310 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476e440 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476e550 2024-02-14T18:17:57.685 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476ec50 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476e7b0 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476e900 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476ea10 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476ed60 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476ee70 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476f420 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476ef80 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476f0b0 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476f1c0 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476f310 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476f9d0 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476f530 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476f640 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476f770 2024-02-14T18:17:57.686 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476f880 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476ffe0 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476fb40 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476fc50 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476fda0 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c476feb0 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4770590 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47700f0 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4770220 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4770330 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4770460 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4770b40 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47706a0 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47707d0 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47708e0 2024-02-14T18:17:57.687 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4770a10 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47710f0 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4770c50 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4770d60 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4770e70 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4770f80 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47716c0 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4771220 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4771350 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4771480 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47717d0 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4771c60 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4771d70 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47718e0 2024-02-14T18:17:57.688 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47719f0 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4771b20 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4771ea0 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4771fd0 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4772580 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47720e0 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47721f0 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4772320 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4772690 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4772b20 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4772c30 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4773200 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47727c0 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4772430 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47728f0 2024-02-14T18:17:57.689 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4772a00 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4772d60 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4772e70 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4772f80 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47730d0 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4773810 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4773370 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47734a0 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47735b0 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4773700 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4773e00 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4773960 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4773a70 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4773bc0 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4773cd0 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47743f0 2024-02-14T18:17:57.690 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4773f50 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4774060 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47741c0 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4774500 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4774970 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4774ae0 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47742d0 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4774610 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4774760 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4774bf0 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4774d00 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4774e50 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4775110 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4775580 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4775690 2024-02-14T18:17:57.691 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4775c60 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47757c0 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4774fe0 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47758f0 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4775a00 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4775b10 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4775d70 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4776250 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4776360 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47688b0 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47689c0 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4776950 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47764b0 2024-02-14T18:17:57.692 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47765e0 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4776710 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4776840 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4776f00 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4776a60 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4776b90 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4776ca0 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4776dd0 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47774d0 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4777030 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4777180 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47772b0 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47775e0 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4777a70 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4777ba0 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47776f0 2024-02-14T18:17:57.693 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4777800 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4777950 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4777cf0 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4777e80 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4778470 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4777fd0 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4775ea0 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4778d80 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47788e0 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4775fd0 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47760e0 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4778a10 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4778b20 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4778ed0 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4779000 2024-02-14T18:17:57.694 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47795f0 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4779150 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47792a0 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47793d0 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47794e0 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4779ba0 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4779700 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4779830 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4779940 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4779a50 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477a170 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4779cd0 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4779e00 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4779f10 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477a040 2024-02-14T18:17:57.695 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477a720 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477a280 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477a3b0 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477a4e0 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477a850 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477acc0 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477adf0 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477b3c0 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477a960 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477a610 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477aa90 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477b510 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477af20 2024-02-14T18:17:57.696 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477b050 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477aba0 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477b160 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477b270 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477b640 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477bab0 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477bc00 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4778580 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47786d0 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477c1f0 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477bd50 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477be80 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477bfb0 2024-02-14T18:17:57.697 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477c0e0 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477c7e0 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477c340 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477c490 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477c5a0 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477c930 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477cde0 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477cef0 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477c6b0 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477ca80 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477cbb0 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477d000 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477d490 2024-02-14T18:17:57.698 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477d5c0 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477b750 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477b880 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477db90 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477d6f0 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477d820 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477d950 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477da80 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477e140 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477dca0 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477ddb0 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477dee0 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477e030 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477e6f0 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477e250 2024-02-14T18:17:57.699 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477e380 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477e490 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477e5e0 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477ecc0 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477e820 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477e950 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477ea60 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477ebb0 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477f290 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477edf0 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477ef40 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477f090 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477f3e0 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477f870 2024-02-14T18:17:57.700 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477f9a0 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477d130 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477d260 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477ff70 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477fad0 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477fc00 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477fd30 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477fe40 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4780540 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47800a0 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47801b0 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4780300 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4780670 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4780b00 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4780c30 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4780430 2024-02-14T18:17:57.701 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47807a0 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47808d0 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4780d60 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47811f0 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4781320 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4780e70 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4780fc0 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47818f0 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4781450 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4781580 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47816b0 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47817e0 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4781ec0 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4781a20 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4781b50 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4781c80 2024-02-14T18:17:57.702 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4781ff0 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47824d0 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782600 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782bd0 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782120 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782250 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782730 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782860 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782380 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782990 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782ac0 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782d00 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4783170 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47832c0 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4783890 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782e10 2024-02-14T18:17:57.703 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4782f60 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47833f0 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4783520 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4783630 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4783760 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4783e80 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47839e0 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4783b10 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4783c40 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4783fb0 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4784440 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4784570 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47840c0 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4784210 2024-02-14T18:17:57.704 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4783d70 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47846a0 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47848c0 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4784eb0 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4784a10 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4784b20 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4784c50 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4784d80 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4785480 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4784fe0 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4785110 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4785240 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4785370 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4785a50 2024-02-14T18:17:57.705 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47855b0 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47856e0 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47857f0 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4785940 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4786020 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4785b80 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4785cb0 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4785dc0 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4785f10 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4786610 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4786170 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47862c0 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47863f0 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4786760 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4786c40 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4786d70 2024-02-14T18:17:57.706 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4786890 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47869c0 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4786af0 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4786ea0 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4787330 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4787480 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477f4f0 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c477f620 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4787a50 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47875b0 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47876e0 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4787810 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4787940 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788020 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4787b80 2024-02-14T18:17:57.707 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4787cd0 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4787e00 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788150 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47885e0 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788710 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4786fb0 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47870e0 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788ce0 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788840 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788970 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4787210 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788aa0 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788bd0 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788e10 2024-02-14T18:17:57.708 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788f80 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4789570 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47890d0 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4789200 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4789330 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4789460 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4789b40 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47896a0 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47897d0 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c47898e0 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4789a30 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478a110 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4789c70 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4789da0 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4789ef0 2024-02-14T18:17:57.709 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478a260 2024-02-14T18:17:57.710 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478a6f0 2024-02-14T18:17:57.710 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478a870 2024-02-14T18:17:57.710 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788260 2024-02-14T18:17:57.710 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4788390 2024-02-14T18:17:57.710 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478ae40 2024-02-14T18:17:57.710 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478a9a0 2024-02-14T18:17:57.710 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478aad0 2024-02-14T18:17:57.710 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478ac00 2024-02-14T18:17:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:57 smithi179 ceph-mon[33633]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:17:57.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:57 smithi179 ceph-mon[33633]: osdmap e756: 8 total, 8 up, 8 in 2024-02-14T18:17:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[39557]: osdmap e757: 8 total, 8 up, 8 in 2024-02-14T18:17:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[39557]: pgmap v1658: 260 pgs: 32 unknown, 228 active+clean; 582 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[30138]: osdmap e757: 8 total, 8 up, 8 in 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:17:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:17:58.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:58 smithi100 ceph-mon[30138]: pgmap v1658: 260 pgs: 32 unknown, 228 active+clean; 582 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:58 smithi179 ceph-mon[33633]: osdmap e757: 8 total, 8 up, 8 in 2024-02-14T18:17:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:17:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:17:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:17:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:17:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:58 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:17:58.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:58 smithi179 ceph-mon[33633]: pgmap v1658: 260 pgs: 32 unknown, 228 active+clean; 582 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:17:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:17:59 smithi100 ceph-mon[30138]: osdmap e758: 8 total, 8 up, 8 in 2024-02-14T18:17:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:17:59 smithi100 ceph-mon[39557]: osdmap e758: 8 total, 8 up, 8 in 2024-02-14T18:17:59.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:17:59 smithi179 ceph-mon[33633]: osdmap e758: 8 total, 8 up, 8 in 2024-02-14T18:18:00.649 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:18:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:18:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:18:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:00 smithi100 ceph-mon[30138]: osdmap e759: 8 total, 8 up, 8 in 2024-02-14T18:18:00.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:00 smithi100 ceph-mon[30138]: pgmap v1661: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:00 smithi100 ceph-mon[39557]: osdmap e759: 8 total, 8 up, 8 in 2024-02-14T18:18:00.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:00 smithi100 ceph-mon[39557]: pgmap v1661: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:00 smithi179 ceph-mon[33633]: osdmap e759: 8 total, 8 up, 8 in 2024-02-14T18:18:00.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:00 smithi179 ceph-mon[33633]: pgmap v1661: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:01 smithi100 ceph-mon[30138]: osdmap e760: 8 total, 8 up, 8 in 2024-02-14T18:18:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:18:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:01 smithi100 ceph-mon[39557]: osdmap e760: 8 total, 8 up, 8 in 2024-02-14T18:18:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:18:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:01 smithi179 ceph-mon[33633]: osdmap e760: 8 total, 8 up, 8 in 2024-02-14T18:18:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:18:02.359 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4 watch_notify: Running main() from gmock_main.cc 2024-02-14T18:18:02.359 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-02-14T18:18:02.359 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [----------] Global test environment set-up. 2024-02-14T18:18:02.359 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: handle_notify cookie 94323778503856 notify_id 3238405341190 notifier_gid 24763 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (1808874 ms) 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: Trying... 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: handle_notify cookie 94323791201136 notify_id 3251290243079 notifier_gid 26213 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: Waiting for 3.000000000s 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: Timed out. 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: Flushing... 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: Flushed... 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7052 ms) 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (1815927 ms total) 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: 2024-02-14T18:18:02.360 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [----------] Global test environment tear-down 2024-02-14T18:18:02.361 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (1815927 ms total) 2024-02-14T18:18:02.361 INFO:tasks.workunit.client.0.smithi100.stdout: watch_notify: [ PASSED ] 2 tests. 2024-02-14T18:18:02.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[39557]: osdmap e761: 8 total, 8 up, 8 in 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[39557]: pgmap v1664: 228 pgs: 32 creating+peering, 196 active+clean; 582 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[39557]: osdmap e762: 8 total, 8 up, 8 in 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[30138]: osdmap e761: 8 total, 8 up, 8 in 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:18:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[30138]: pgmap v1664: 228 pgs: 32 creating+peering, 196 active+clean; 582 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:02.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T18:18:02.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:02 smithi100 ceph-mon[30138]: osdmap e762: 8 total, 8 up, 8 in 2024-02-14T18:18:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-14T18:18:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:02 smithi179 ceph-mon[33633]: osdmap e761: 8 total, 8 up, 8 in 2024-02-14T18:18:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-14T18:18:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:18:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:02 smithi179 ceph-mon[33633]: pgmap v1664: 228 pgs: 32 creating+peering, 196 active+clean; 582 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:02 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-14T18:18:02.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:02 smithi179 ceph-mon[33633]: osdmap e762: 8 total, 8 up, 8 in 2024-02-14T18:18:03.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:18:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:18:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app2"}]: dispatch 2024-02-14T18:18:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:18:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:03 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T18:18:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:03 smithi179 ceph-mon[33633]: osdmap e763: 8 total, 8 up, 8 in 2024-02-14T18:18:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-14T18:18:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:03 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T18:18:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app2"}]: dispatch 2024-02-14T18:18:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:18:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T18:18:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[39557]: osdmap e763: 8 total, 8 up, 8 in 2024-02-14T18:18:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-14T18:18:03.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T18:18:03.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app2"}]: dispatch 2024-02-14T18:18:03.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-14T18:18:03.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:03.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi100-79744-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-14T18:18:03.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[30138]: osdmap e763: 8 total, 8 up, 8 in 2024-02-14T18:18:03.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-14T18:18:03.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:03 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-14T18:18:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:04 smithi100 ceph-mon[30138]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:04 smithi100 ceph-mon[30138]: pgmap v1667: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T18:18:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:04 smithi100 ceph-mon[30138]: osdmap e764: 8 total, 8 up, 8 in 2024-02-14T18:18:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T18:18:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:04 smithi100 ceph-mon[39557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:04 smithi100 ceph-mon[39557]: pgmap v1667: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:04.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T18:18:04.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:04 smithi100 ceph-mon[39557]: osdmap e764: 8 total, 8 up, 8 in 2024-02-14T18:18:04.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T18:18:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:04 smithi179 ceph-mon[33633]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:04 smithi179 ceph-mon[33633]: pgmap v1667: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-14T18:18:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:04 smithi179 ceph-mon[33633]: osdmap e764: 8 total, 8 up, 8 in 2024-02-14T18:18:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-14T18:18:06.361 INFO:tasks.workunit.client.0.smithi100.stdout:78ad30 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478b410 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478af70 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478b0c0 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478b1f0 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478b540 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478b9d0 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478bb00 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478a370 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478a4a0 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478c0d0 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478bc30 2024-02-14T18:18:06.362 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478bd60 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478be90 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478bfc0 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478c6a0 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478c200 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478c330 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478c440 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478c590 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478cc70 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478c7d0 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478c900 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478ca10 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478cb60 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478d260 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478cdc0 2024-02-14T18:18:06.363 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478cf10 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478d040 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478d3b0 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478d840 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478d150 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478b650 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478b780 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478de10 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478d970 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478da80 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478dbd0 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478df60 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478e410 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478e520 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478dd00 2024-02-14T18:18:06.364 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478e090 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478e1e0 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478e650 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478e780 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478ed70 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478e8d0 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478ea20 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478eb50 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478eec0 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478f350 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478f480 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478d4e0 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478d610 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478ec60 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478d720 2024-02-14T18:18:06.365 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478fa50 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478f5b0 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478f700 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478f810 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478f920 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c4790040 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478fba0 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: waiting 0x5648c478fcd0 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: done. 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (1810459 ms) 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (1810459 ms total) 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (1 ms) 2024-02-14T18:18:06.366 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-14T18:18:06.367 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.Exec (14 ms) 2024-02-14T18:18:06.367 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-14T18:18:06.367 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (5 ms) 2024-02-14T18:18:06.367 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-14T18:18:06.367 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (5 ms) 2024-02-14T18:18:06.367 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-14T18:18:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T18:18:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:06 smithi100 ceph-mon[30138]: osdmap e765: 8 total, 8 up, 8 in 2024-02-14T18:18:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T18:18:06.652 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:06 smithi100 ceph-mon[30138]: pgmap v1670: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:18:06.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T18:18:06.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:06 smithi100 ceph-mon[39557]: osdmap e765: 8 total, 8 up, 8 in 2024-02-14T18:18:06.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T18:18:06.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:06 smithi100 ceph-mon[39557]: pgmap v1670: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:18:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-14T18:18:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:06 smithi179 ceph-mon[33633]: osdmap e765: 8 total, 8 up, 8 in 2024-02-14T18:18:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1"}]: dispatch 2024-02-14T18:18:06.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:06 smithi179 ceph-mon[33633]: pgmap v1670: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:18:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:07 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1"}]': finished 2024-02-14T18:18:07.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:07 smithi100 ceph-mon[30138]: osdmap e766: 8 total, 8 up, 8 in 2024-02-14T18:18:07.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:07 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1"}]': finished 2024-02-14T18:18:07.653 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:07 smithi100 ceph-mon[39557]: osdmap e766: 8 total, 8 up, 8 in 2024-02-14T18:18:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:07 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4094370203' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi100-79744-1","app":"app1","key":"key1"}]': finished 2024-02-14T18:18:07.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:07 smithi179 ceph-mon[33633]: osdmap e766: 8 total, 8 up, 8 in 2024-02-14T18:18:08.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:08 smithi100 ceph-mon[30138]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 880 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-02-14T18:18:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:08 smithi100 ceph-mon[39557]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 880 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-02-14T18:18:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:08 smithi179 ceph-mon[33633]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 880 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-02-14T18:18:09.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:09 smithi100 ceph-mon[39557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:09 smithi100 ceph-mon[30138]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:09.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:09 smithi179 ceph-mon[33633]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:10.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:10 smithi100 ceph-mon[30138]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 880 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 853 B/s wr, 1 op/s 2024-02-14T18:18:10.651 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:18:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:18:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:18:10.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:10 smithi100 ceph-mon[39557]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 880 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 853 B/s wr, 1 op/s 2024-02-14T18:18:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:10 smithi179 ceph-mon[33633]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 880 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 853 B/s wr, 1 op/s 2024-02-14T18:18:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:13 smithi179 ceph-mon[33633]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 649 B/s wr, 2 op/s 2024-02-14T18:18:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:13.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:18:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:18:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:13 smithi100 ceph-mon[30138]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 649 B/s wr, 2 op/s 2024-02-14T18:18:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:13 smithi100 ceph-mon[39557]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 649 B/s wr, 2 op/s 2024-02-14T18:18:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:15 smithi179 ceph-mon[33633]: pgmap v1675: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 576 B/s wr, 1 op/s 2024-02-14T18:18:15.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:15 smithi100 ceph-mon[30138]: pgmap v1675: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 576 B/s wr, 1 op/s 2024-02-14T18:18:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:15 smithi100 ceph-mon[39557]: pgmap v1675: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 576 B/s wr, 1 op/s 2024-02-14T18:18:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:17 smithi179 ceph-mon[33633]: pgmap v1676: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T18:18:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:17 smithi100 ceph-mon[30138]: pgmap v1676: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T18:18:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:17 smithi100 ceph-mon[39557]: pgmap v1676: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-14T18:18:19.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:19 smithi179 ceph-mon[33633]: pgmap v1677: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-02-14T18:18:19.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:19 smithi100 ceph-mon[39557]: pgmap v1677: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-02-14T18:18:19.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:19 smithi100 ceph-mon[30138]: pgmap v1677: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-02-14T18:18:20.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:18:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:18:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:18:21.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:21 smithi179 ceph-mon[33633]: pgmap v1678: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:21.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:21 smithi100 ceph-mon[30138]: pgmap v1678: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:21.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:21 smithi100 ceph-mon[39557]: pgmap v1678: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:22.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:22 smithi100 ceph-mon[30138]: pgmap v1679: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:22.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:22 smithi100 ceph-mon[39557]: pgmap v1679: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:22.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:22 smithi179 ceph-mon[33633]: pgmap v1679: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:23.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:18:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:18:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4983 ms) 2024-02-14T18:18:24.607 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-14T18:18:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-14T18:18:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-14T18:18:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-14T18:18:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-14T18:18:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (17583 ms) 2024-02-14T18:18:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] 8 tests from LibRadosMisc (22591 ms total) 2024-02-14T18:18:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: 2024-02-14T18:18:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [----------] Global test environment tear-down 2024-02-14T18:18:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (1838519 ms total) 2024-02-14T18:18:24.608 INFO:tasks.workunit.client.0.smithi100.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-14T18:18:24.615 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.615 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79957 2024-02-14T18:18:24.615 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79957 2024-02-14T18:18:24.615 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.615 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80449 2024-02-14T18:18:24.615 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80449 2024-02-14T18:18:24.615 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.615 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80480 2024-02-14T18:18:24.615 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80480 2024-02-14T18:18:24.615 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79668 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79668 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80322 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80322 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80197 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80197 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79541 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79541 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79604 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79604 2024-02-14T18:18:24.616 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80230 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80230 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79532 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79532 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79552 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79552 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80608 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80608 2024-02-14T18:18:24.617 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.618 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79776 2024-02-14T18:18:24.618 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79776 2024-02-14T18:18:24.618 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.618 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80752 2024-02-14T18:18:24.618 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80752 2024-02-14T18:18:24.618 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79923 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79923 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80365 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80365 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80827 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80827 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79821 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79821 2024-02-14T18:18:24.619 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80278 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80278 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80545 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80545 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80338 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80338 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80686 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80686 2024-02-14T18:18:24.620 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80158 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80158 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79729 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79729 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80810 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80810 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80025 2024-02-14T18:18:24.621 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80025 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79525 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79525 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79586 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79586 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79983 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79983 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=79626 2024-02-14T18:18:24.622 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 79626 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80038 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80038 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80052 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80052 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80497 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80497 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ for t in "${!pids[@]}" 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=80852 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 80852 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ exit 0 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ cleanup 2024-02-14T18:18:24.623 INFO:tasks.workunit.client.0.smithi100.stderr:+ pkill -P 79520 2024-02-14T18:18:24.631 INFO:tasks.workunit.client.0.smithi100.stderr:+ true 2024-02-14T18:18:24.632 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-14T18:18:24.632 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-14T18:18:24.665 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-02-14T18:18:24.666 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-02-14T18:18:24.666 DEBUG:teuthology.orchestra.run.smithi100:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8727ce939fc044a98f215e29696bb1433f5758d1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-02-14T18:18:24.724 INFO:tasks.workunit.client.0.smithi100.stderr:++ uuidgen 2024-02-14T18:18:24.736 INFO:tasks.workunit.client.0.smithi100.stderr:+ p=8892ea91-15b7-4906-9324-dc08ab1d48a5 2024-02-14T18:18:24.736 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool create 8892ea91-15b7-4906-9324-dc08ab1d48a5 12 2024-02-14T18:18:24.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.817+0000 7f5c411ac640 1 -- 172.21.15.100:0/1958010240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c3c1013c0 msgr2=0x7f5c3c1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:24.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.817+0000 7f5c411ac640 1 --2- 172.21.15.100:0/1958010240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c3c1013c0 0x7f5c3c1017a0 secure :-1 s=READY pgs=3070 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009890 tx=0x7f5c2c02bb70 comp rx=0 tx=0).stop 2024-02-14T18:18:24.820 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.817+0000 7f5c411ac640 1 -- 172.21.15.100:0/1958010240 shutdown_connections 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.817+0000 7f5c411ac640 1 --2- 172.21.15.100:0/1958010240 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c3c106490 0x7f5c3c10e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.817+0000 7f5c411ac640 1 --2- 172.21.15.100:0/1958010240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c3c101d70 0x7f5c3c105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.817+0000 7f5c411ac640 1 --2- 172.21.15.100:0/1958010240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c3c1013c0 0x7f5c3c1017a0 unknown :-1 s=CLOSED pgs=3070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.817+0000 7f5c411ac640 1 -- 172.21.15.100:0/1958010240 >> 172.21.15.100:0/1958010240 conn(0x7f5c3c0fcc40 msgr2=0x7f5c3c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c411ac640 1 -- 172.21.15.100:0/1958010240 shutdown_connections 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c411ac640 1 -- 172.21.15.100:0/1958010240 wait complete. 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c411ac640 1 Processor -- start 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c411ac640 1 -- start start 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c411ac640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c3c1013c0 0x7f5c3c196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c411ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c3c106490 0x7f5c3c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c411ac640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c3c19b740 0x7f5c3c19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c411ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c19c130 con 0x7f5c3c106490 2024-02-14T18:18:24.821 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c411ac640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c19c2a0 con 0x7f5c3c1013c0 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c411ac640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c19c410 con 0x7f5c3c19b740 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c3b59e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c3c19b740 0x7f5c3c19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.818+0000 7f5c33fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c3c106490 0x7f5c3c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c3b59e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c3c19b740 0x7f5c3c19bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:47032/0 (socket says 172.21.15.100:47032) 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c3b59e640 1 -- 172.21.15.100:0/3651411794 learned_addr learned my addr 172.21.15.100:0/3651411794 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c33fff640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c3c106490 0x7f5c3c1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46700/0 (socket says 172.21.15.100:46700) 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c3ad9d640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c3c1013c0 0x7f5c3c196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c3b59e640 1 -- 172.21.15.100:0/3651411794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c3c1013c0 msgr2=0x7f5c3c196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c3b59e640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c3c1013c0 0x7f5c3c196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c3b59e640 1 -- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c3c106490 msgr2=0x7f5c3c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c3b59e640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c3c106490 0x7f5c3c1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c3b59e640 1 -- 172.21.15.100:0/3651411794 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34009900 con 0x7f5c3c19b740 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c33fff640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c3c106490 0x7f5c3c1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:18:24.822 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c3b59e640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c3c19b740 0x7f5c3c19bbf0 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400efc0 tx=0x7f5c3400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:18:24.823 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c38d99640 1 -- 172.21.15.100:0/3651411794 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34003970 con 0x7f5c3c19b740 2024-02-14T18:18:24.823 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c009490 con 0x7f5c3c19b740 2024-02-14T18:18:24.823 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c38d99640 1 -- 172.21.15.100:0/3651411794 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5c34004020 con 0x7f5c3c19b740 2024-02-14T18:18:24.823 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c38d99640 1 -- 172.21.15.100:0/3651411794 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400b830 con 0x7f5c3c19b740 2024-02-14T18:18:24.823 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.819+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3c1a3cc0 con 0x7f5c3c19b740 2024-02-14T18:18:24.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.820+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3c0684b0 con 0x7f5c3c19b740 2024-02-14T18:18:24.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.820+0000 7f5c38d99640 1 -- 172.21.15.100:0/3651411794 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f5c3400a040 con 0x7f5c3c19b740 2024-02-14T18:18:24.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.821+0000 7f5c38d99640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f5c2806da30 0x7f5c2806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:24.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.821+0000 7f5c3ad9d640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f5c2806da30 0x7f5c2806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:24.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.821+0000 7f5c38d99640 1 -- 172.21.15.100:0/3651411794 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(767..767 src has 254..767) v4 ==== 8659+0+0 (secure 0 0 0) 0x7f5c3408dde0 con 0x7f5c3c19b740 2024-02-14T18:18:24.824 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.821+0000 7f5c3ad9d640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f5c2806da30 0x7f5c2806fef0 secure :-1 s=READY pgs=4080 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005940 tx=0x7f5c2c0058b0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:18:24.826 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:24.826+0000 7f5c38d99640 1 -- 172.21.15.100:0/3651411794 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c34056d20 con 0x7f5c3c19b740 2024-02-14T18:18:24.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:24.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:24 smithi100 ceph-mon[30138]: pgmap v1680: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:24.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:24.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:24 smithi100 ceph-mon[39557]: pgmap v1680: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:25.001 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.000+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12} v 0) v1 -- 0x7f5c3c101d70 con 0x7f5c3c19b740 2024-02-14T18:18:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:25.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:24 smithi179 ceph-mon[33633]: pgmap v1680: 196 pgs: 196 active+clean; 583 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:25.596 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.595+0000 7f5c38d99640 1 -- 172.21.15.100:0/3651411794 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]=0 pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' created v768) v1 ==== 176+0+0 (secure 0 0 0) 0x7f5c3405a3e0 con 0x7f5c3c19b740 2024-02-14T18:18:25.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.701+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12} v 0) v1 -- 0x7f5c3c110730 con 0x7f5c3c19b740 2024-02-14T18:18:25.702 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.702+0000 7f5c38d99640 1 -- 172.21.15.100:0/3651411794 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]=0 pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' already exists v768) v1 ==== 183+0+0 (secure 0 0 0) 0x7f5c3405a5c0 con 0x7f5c3c19b740 2024-02-14T18:18:25.703 INFO:tasks.workunit.client.0.smithi100.stderr:pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' already exists 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f5c2806da30 msgr2=0x7f5c2806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f5c2806da30 0x7f5c2806fef0 secure :-1 s=READY pgs=4080 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005940 tx=0x7f5c2c0058b0 comp rx=0 tx=0).stop 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c3c19b740 msgr2=0x7f5c3c19bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c3c19b740 0x7f5c3c19bbf0 secure :-1 s=READY pgs=3113 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400efc0 tx=0x7f5c3400c800 comp rx=0 tx=0).stop 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 shutdown_connections 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f5c2806da30 0x7f5c2806fef0 unknown :-1 s=CLOSED pgs=4080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f5c3c19b740 0x7f5c3c19bbf0 unknown :-1 s=CLOSED pgs=3113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c3c106490 0x7f5c3c1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 --2- 172.21.15.100:0/3651411794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c3c1013c0 0x7f5c3c196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 >> 172.21.15.100:0/3651411794 conn(0x7f5c3c0fcc40 msgr2=0x7f5c3c0fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:18:25.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 shutdown_connections 2024-02-14T18:18:25.706 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.704+0000 7f5c411ac640 1 -- 172.21.15.100:0/3651411794 wait complete. 2024-02-14T18:18:25.715 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 8892ea91-15b7-4906-9324-dc08ab1d48a5 max_objects 10 2024-02-14T18:18:25.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.797+0000 7fcb452c9640 1 -- 172.21.15.100:0/692758792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb40101d60 msgr2=0x7fcb40105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:25.800 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.797+0000 7fcb452c9640 1 --2- 172.21.15.100:0/692758792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb40101d60 0x7fcb40105da0 secure :-1 s=READY pgs=3071 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009930 tx=0x7fcb3402bc30 comp rx=0 tx=0).stop 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.797+0000 7fcb452c9640 1 -- 172.21.15.100:0/692758792 shutdown_connections 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.797+0000 7fcb452c9640 1 --2- 172.21.15.100:0/692758792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb40106480 0x7fcb4010e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.797+0000 7fcb452c9640 1 --2- 172.21.15.100:0/692758792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb40101d60 0x7fcb40105da0 unknown :-1 s=CLOSED pgs=3071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.797+0000 7fcb452c9640 1 --2- 172.21.15.100:0/692758792 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb401013b0 0x7fcb40101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.797+0000 7fcb452c9640 1 -- 172.21.15.100:0/692758792 >> 172.21.15.100:0/692758792 conn(0x7fcb400fcc10 msgr2=0x7fcb400ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.797+0000 7fcb452c9640 1 -- 172.21.15.100:0/692758792 shutdown_connections 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.797+0000 7fcb452c9640 1 -- 172.21.15.100:0/692758792 wait complete. 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb452c9640 1 Processor -- start 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb452c9640 1 -- start start 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb452c9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb401013b0 0x7fcb40196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb452c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb40106480 0x7fcb401972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb452c9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4019b640 0x7fcb4019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb452c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4019c030 con 0x7fcb40106480 2024-02-14T18:18:25.801 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb452c9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4019c1a0 con 0x7fcb401013b0 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb452c9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4019c310 con 0x7fcb4019b640 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb3f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4019b640 0x7fcb4019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb3e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb40106480 0x7fcb401972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb3f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4019b640 0x7fcb4019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:47048/0 (socket says 172.21.15.100:47048) 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb3e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb40106480 0x7fcb401972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46718/0 (socket says 172.21.15.100:46718) 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.798+0000 7fcb3f7fe640 1 -- 172.21.15.100:0/1501026818 learned_addr learned my addr 172.21.15.100:0/1501026818 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb3effd640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb401013b0 0x7fcb40196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb3f7fe640 1 -- 172.21.15.100:0/1501026818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb401013b0 msgr2=0x7fcb40196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb3f7fe640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb401013b0 0x7fcb40196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb3f7fe640 1 -- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb40106480 msgr2=0x7fcb401972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb3f7fe640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb40106480 0x7fcb401972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb3f7fe640 1 -- 172.21.15.100:0/1501026818 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb30009490 con 0x7fcb4019b640 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb3e7fc640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb40106480 0x7fcb401972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:18:25.802 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb3f7fe640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4019b640 0x7fcb4019baf0 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800eaa0 tx=0x7fcb3800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:18:25.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb27fff640 1 -- 172.21.15.100:0/1501026818 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800cdb0 con 0x7fcb4019b640 2024-02-14T18:18:25.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb34009560 con 0x7fcb4019b640 2024-02-14T18:18:25.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb27fff640 1 -- 172.21.15.100:0/1501026818 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcb38005020 con 0x7fcb4019b640 2024-02-14T18:18:25.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb27fff640 1 -- 172.21.15.100:0/1501026818 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb380116d0 con 0x7fcb4019b640 2024-02-14T18:18:25.803 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.799+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb401a3b90 con 0x7fcb4019b640 2024-02-14T18:18:25.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.800+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb40068060 con 0x7fcb4019b640 2024-02-14T18:18:25.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.801+0000 7fcb27fff640 1 -- 172.21.15.100:0/1501026818 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7fcb38004a40 con 0x7fcb4019b640 2024-02-14T18:18:25.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.801+0000 7fcb27fff640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcb2806da30 0x7fcb2806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:25.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.801+0000 7fcb3effd640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcb2806da30 0x7fcb2806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:25.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.801+0000 7fcb27fff640 1 -- 172.21.15.100:0/1501026818 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(768..768 src has 254..768) v4 ==== 9034+0+0 (secure 0 0 0) 0x7fcb3808e060 con 0x7fcb4019b640 2024-02-14T18:18:25.804 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.801+0000 7fcb3effd640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcb2806da30 0x7fcb2806fef0 secure :-1 s=READY pgs=4081 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009d60 tx=0x7fcb30005f90 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:18:25.806 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.806+0000 7fcb27fff640 1 -- 172.21.15.100:0/1501026818 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb38056ee0 con 0x7fcb4019b640 2024-02-14T18:18:25.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:25 smithi100 ceph-mon[30138]: osdmap e767: 8 total, 8 up, 8 in 2024-02-14T18:18:25.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:25 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3651411794' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:25.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:25 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:25.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:25 smithi100 ceph-mon[39557]: osdmap e767: 8 total, 8 up, 8 in 2024-02-14T18:18:25.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:25 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3651411794' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:25.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:25 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:25.982 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:25.982+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fcb40101d60 con 0x7fcb4019b640 2024-02-14T18:18:26.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:25 smithi179 ceph-mon[33633]: osdmap e767: 8 total, 8 up, 8 in 2024-02-14T18:18:26.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:25 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3651411794' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:26.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:25 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:26.606 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:26.605+0000 7fcb27fff640 1 -- 172.21.15.100:0/1501026818 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v769) v1 ==== 223+0+0 (secure 0 0 0) 0x7fcb3805a5a0 con 0x7fcb4019b640 2024-02-14T18:18:26.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:26.715+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fcb40101f70 con 0x7fcb4019b640 2024-02-14T18:18:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]': finished 2024-02-14T18:18:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[39557]: osdmap e768: 8 total, 8 up, 8 in 2024-02-14T18:18:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3651411794' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1501026818' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:26.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[39557]: pgmap v1683: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 383 B/s rd, 0 op/s 2024-02-14T18:18:26.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]': finished 2024-02-14T18:18:26.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[30138]: osdmap e768: 8 total, 8 up, 8 in 2024-02-14T18:18:26.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3651411794' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:26.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:26.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1501026818' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:26.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:26.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:26 smithi100 ceph-mon[30138]: pgmap v1683: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 383 B/s rd, 0 op/s 2024-02-14T18:18:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]': finished 2024-02-14T18:18:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:26 smithi179 ceph-mon[33633]: osdmap e768: 8 total, 8 up, 8 in 2024-02-14T18:18:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3651411794' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pg_num": 12}]: dispatch 2024-02-14T18:18:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:26 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1501026818' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:27.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:26 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:27.072 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:26 smithi179 ceph-mon[33633]: pgmap v1683: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 876 MiB used, 714 GiB / 715 GiB avail; 383 B/s rd, 0 op/s 2024-02-14T18:18:27.611 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.610+0000 7fcb27fff640 1 -- 172.21.15.100:0/1501026818 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v770) v1 ==== 223+0+0 (secure 0 0 0) 0x7fcb3801f020 con 0x7fcb4019b640 2024-02-14T18:18:27.611 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_objects = 10 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.612+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcb2806da30 msgr2=0x7fcb2806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.612+0000 7fcb452c9640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcb2806da30 0x7fcb2806fef0 secure :-1 s=READY pgs=4081 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009d60 tx=0x7fcb30005f90 comp rx=0 tx=0).stop 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.612+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4019b640 msgr2=0x7fcb4019baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.612+0000 7fcb452c9640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4019b640 0x7fcb4019baf0 secure :-1 s=READY pgs=3114 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800eaa0 tx=0x7fcb3800ef60 comp rx=0 tx=0).stop 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.612+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 shutdown_connections 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.613+0000 7fcb452c9640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fcb2806da30 0x7fcb2806fef0 unknown :-1 s=CLOSED pgs=4081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.613+0000 7fcb452c9640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fcb4019b640 0x7fcb4019baf0 unknown :-1 s=CLOSED pgs=3114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.613+0000 7fcb452c9640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb40106480 0x7fcb401972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.613+0000 7fcb452c9640 1 --2- 172.21.15.100:0/1501026818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb401013b0 0x7fcb40196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.613+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 >> 172.21.15.100:0/1501026818 conn(0x7fcb400fcc10 msgr2=0x7fcb400fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.613+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 shutdown_connections 2024-02-14T18:18:27.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.613+0000 7fcb452c9640 1 -- 172.21.15.100:0/1501026818 wait complete. 2024-02-14T18:18:27.625 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool application enable 8892ea91-15b7-4906-9324-dc08ab1d48a5 rados 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.711+0000 7f91e10e2640 1 -- 172.21.15.100:0/13495544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc10b7c0 msgr2=0x7f91dc10dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.711+0000 7f91e10e2640 1 --2- 172.21.15.100:0/13495544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc10b7c0 0x7f91dc10dbf0 secure :-1 s=READY pgs=3072 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009ca0 tx=0x7f91d402c180 comp rx=0 tx=0).stop 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.711+0000 7f91e10e2640 1 -- 172.21.15.100:0/13495544 shutdown_connections 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.711+0000 7f91e10e2640 1 --2- 172.21.15.100:0/13495544 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc10b7c0 0x7f91dc10dbf0 unknown :-1 s=CLOSED pgs=3072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.711+0000 7f91e10e2640 1 --2- 172.21.15.100:0/13495544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91dc102d10 0x7f91dc10b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.711+0000 7f91e10e2640 1 --2- 172.21.15.100:0/13495544 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91dc1023f0 0x7f91dc1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.711+0000 7f91e10e2640 1 -- 172.21.15.100:0/13495544 >> 172.21.15.100:0/13495544 conn(0x7f91dc077170 msgr2=0x7f91dc077580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.711+0000 7f91e10e2640 1 -- 172.21.15.100:0/13495544 shutdown_connections 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.711+0000 7f91e10e2640 1 -- 172.21.15.100:0/13495544 wait complete. 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.712+0000 7f91e10e2640 1 Processor -- start 2024-02-14T18:18:27.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.712+0000 7f91e10e2640 1 -- start start 2024-02-14T18:18:27.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.712+0000 7f91e10e2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91dc1023f0 0x7f91dc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:27.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.712+0000 7f91e10e2640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91dc102d10 0x7f91dc19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:27.716 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.712+0000 7f91e10e2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc10b7c0 0x7f91dc196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91e10e2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc1972b0 con 0x7f91dc10b7c0 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91e10e2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc197420 con 0x7f91dc1023f0 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91e10e2640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc1975e0 con 0x7f91dc102d10 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91db59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc10b7c0 0x7f91dc196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91da59c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91dc102d10 0x7f91dc19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91db59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc10b7c0 0x7f91dc196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46738/0 (socket says 172.21.15.100:46738) 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91db59e640 1 -- 172.21.15.100:0/2874804831 learned_addr learned my addr 172.21.15.100:0/2874804831 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91db59e640 1 -- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91dc102d10 msgr2=0x7f91dc19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91db59e640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91dc102d10 0x7f91dc19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91db59e640 1 -- 172.21.15.100:0/2874804831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91dc1023f0 msgr2=0x7f91dc19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91dad9d640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91dc1023f0 0x7f91dc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91db59e640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91dc1023f0 0x7f91dc19ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91db59e640 1 -- 172.21.15.100:0/2874804831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d4009900 con 0x7f91dc10b7c0 2024-02-14T18:18:27.717 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91da59c640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91dc102d10 0x7f91dc19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91db59e640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc10b7c0 0x7f91dc196b20 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7f91d402c790 tx=0x7f91d40347b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91c3fff640 1 -- 172.21.15.100:0/2874804831 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4003d80 con 0x7f91dc10b7c0 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc197860 con 0x7f91dc10b7c0 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91c3fff640 1 -- 172.21.15.100:0/2874804831 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f91d40044c0 con 0x7f91dc10b7c0 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91c3fff640 1 -- 172.21.15.100:0/2874804831 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4032a80 con 0x7f91dc10b7c0 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.713+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc1a39e0 con 0x7f91dc10b7c0 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.714+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc06a0e0 con 0x7f91dc10b7c0 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.715+0000 7f91c3fff640 1 -- 172.21.15.100:0/2874804831 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f91d403b040 con 0x7f91dc10b7c0 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.715+0000 7f91c3fff640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f91c406da30 0x7f91c406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.715+0000 7f91dad9d640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f91c406da30 0x7f91c406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.715+0000 7f91c3fff640 1 -- 172.21.15.100:0/2874804831 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(770..770 src has 254..770) v4 ==== 9034+0+0 (secure 0 0 0) 0x7f91d40b0a20 con 0x7f91dc10b7c0 2024-02-14T18:18:27.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.716+0000 7f91dad9d640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f91c406da30 0x7f91c406fef0 secure :-1 s=READY pgs=4082 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009aa0 tx=0x7f91d0005f90 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:18:27.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.720+0000 7f91c3fff640 1 -- 172.21.15.100:0/2874804831 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91d40797f0 con 0x7f91dc10b7c0 2024-02-14T18:18:27.897 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:27.896+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"} v 0) v1 -- 0x7f91dc197fe0 con 0x7f91dc10b7c0 2024-02-14T18:18:27.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:27 smithi100 ceph-mon[30138]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:27.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:18:27.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:27 smithi100 ceph-mon[30138]: osdmap e769: 8 total, 8 up, 8 in 2024-02-14T18:18:27.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:27 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1501026818' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:27.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:27 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:27.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:27 smithi100 ceph-mon[39557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:27.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:18:27.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:27 smithi100 ceph-mon[39557]: osdmap e769: 8 total, 8 up, 8 in 2024-02-14T18:18:27.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:27 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1501026818' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:27.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:27 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:27 smithi179 ceph-mon[33633]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:18:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:27 smithi179 ceph-mon[33633]: osdmap e769: 8 total, 8 up, 8 in 2024-02-14T18:18:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:27 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1501026818' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:28.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:27 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:18:28.610 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:28.609+0000 7f91c3fff640 1 -- 172.21.15.100:0/2874804831 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]=0 enabled application 'rados' on pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' v771) v1 ==== 213+0+0 (secure 0 0 0) 0x7f91d407ceb0 con 0x7f91dc10b7c0 2024-02-14T18:18:28.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:28.720+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"} v 0) v1 -- 0x7f91dc110980 con 0x7f91dc10b7c0 2024-02-14T18:18:28.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:28 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:18:28.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:28 smithi100 ceph-mon[30138]: osdmap e770: 8 total, 8 up, 8 in 2024-02-14T18:18:28.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:28 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]: dispatch 2024-02-14T18:18:28.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:28.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:28 smithi100 ceph-mon[30138]: pgmap v1686: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:18:28.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:28 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:18:28.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:28 smithi100 ceph-mon[39557]: osdmap e770: 8 total, 8 up, 8 in 2024-02-14T18:18:28.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:28 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]: dispatch 2024-02-14T18:18:28.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:28.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:28 smithi100 ceph-mon[39557]: pgmap v1686: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:18:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:28 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:18:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:28 smithi179 ceph-mon[33633]: osdmap e770: 8 total, 8 up, 8 in 2024-02-14T18:18:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:28 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]: dispatch 2024-02-14T18:18:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:29.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:28 smithi179 ceph-mon[33633]: pgmap v1686: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-14T18:18:29.612 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.610+0000 7f91c3fff640 1 -- 172.21.15.100:0/2874804831 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]=0 enabled application 'rados' on pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' v772) v1 ==== 213+0+0 (secure 0 0 0) 0x7f91d4043020 con 0x7f91dc10b7c0 2024-02-14T18:18:29.612 INFO:tasks.workunit.client.0.smithi100.stderr:enabled application 'rados' on pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' 2024-02-14T18:18:29.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.613+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f91c406da30 msgr2=0x7f91c406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:29.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.613+0000 7f91e10e2640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f91c406da30 0x7f91c406fef0 secure :-1 s=READY pgs=4082 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009aa0 tx=0x7f91d0005f90 comp rx=0 tx=0).stop 2024-02-14T18:18:29.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.613+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc10b7c0 msgr2=0x7f91dc196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:18:29.614 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.613+0000 7f91e10e2640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc10b7c0 0x7f91dc196b20 secure :-1 s=READY pgs=3073 cs=0 l=1 rev1=1 crypto rx=0x7f91d402c790 tx=0x7f91d40347b0 comp rx=0 tx=0).stop 2024-02-14T18:18:29.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.613+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 shutdown_connections 2024-02-14T18:18:29.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.613+0000 7f91e10e2640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f91c406da30 0x7f91c406fef0 unknown :-1 s=CLOSED pgs=4082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:29.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.613+0000 7f91e10e2640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91dc10b7c0 0x7f91dc196b20 unknown :-1 s=CLOSED pgs=3073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:29.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.613+0000 7f91e10e2640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f91dc102d10 0x7f91dc19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:29.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.613+0000 7f91e10e2640 1 --2- 172.21.15.100:0/2874804831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f91dc1023f0 0x7f91dc19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:18:29.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.613+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 >> 172.21.15.100:0/2874804831 conn(0x7f91dc077170 msgr2=0x7f91dc0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:18:29.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.614+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 shutdown_connections 2024-02-14T18:18:29.615 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:18:29.614+0000 7f91e10e2640 1 -- 172.21.15.100:0/2874804831 wait complete. 2024-02-14T18:18:29.629 INFO:tasks.workunit.client.0.smithi100.stderr:++ seq 1 10 2024-02-14T18:18:29.637 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:18:29.637 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put obj1 /etc/passwd 2024-02-14T18:18:29.674 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:18:29.674 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put obj2 /etc/passwd 2024-02-14T18:18:29.711 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:18:29.711 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put obj3 /etc/passwd 2024-02-14T18:18:29.747 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:18:29.747 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put obj4 /etc/passwd 2024-02-14T18:18:29.782 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:18:29.782 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put obj5 /etc/passwd 2024-02-14T18:18:29.816 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:18:29.816 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put obj6 /etc/passwd 2024-02-14T18:18:29.851 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:18:29.851 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put obj7 /etc/passwd 2024-02-14T18:18:29.885 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:18:29.885 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put obj8 /etc/passwd 2024-02-14T18:18:29.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]': finished 2024-02-14T18:18:29.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:29 smithi100 ceph-mon[30138]: osdmap e771: 8 total, 8 up, 8 in 2024-02-14T18:18:29.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:29 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]: dispatch 2024-02-14T18:18:29.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]': finished 2024-02-14T18:18:29.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:29 smithi100 ceph-mon[39557]: osdmap e771: 8 total, 8 up, 8 in 2024-02-14T18:18:29.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:29 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]: dispatch 2024-02-14T18:18:29.919 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:18:29.919 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put obj9 /etc/passwd 2024-02-14T18:18:29.953 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:18:29.954 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put obj10 /etc/passwd 2024-02-14T18:18:29.989 INFO:tasks.workunit.client.0.smithi100.stderr:+ sleep 30 2024-02-14T18:18:30.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]': finished 2024-02-14T18:18:30.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:29 smithi179 ceph-mon[33633]: osdmap e771: 8 total, 8 up, 8 in 2024-02-14T18:18:30.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:29 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]: dispatch 2024-02-14T18:18:30.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:18:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:18:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:18:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:30 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]': finished 2024-02-14T18:18:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:30 smithi100 ceph-mon[30138]: osdmap e772: 8 total, 8 up, 8 in 2024-02-14T18:18:30.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:30 smithi100 ceph-mon[30138]: pgmap v1689: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:30.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:30 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]': finished 2024-02-14T18:18:30.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:30 smithi100 ceph-mon[39557]: osdmap e772: 8 total, 8 up, 8 in 2024-02-14T18:18:30.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:30 smithi100 ceph-mon[39557]: pgmap v1689: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:30 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2874804831' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "app": "rados"}]': finished 2024-02-14T18:18:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:30 smithi179 ceph-mon[33633]: osdmap e772: 8 total, 8 up, 8 in 2024-02-14T18:18:31.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:30 smithi179 ceph-mon[33633]: pgmap v1689: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 877 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:33.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:33 smithi179 ceph-mon[33633]: pgmap v1690: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 908 B/s rd, 21 KiB/s wr, 2 op/s 2024-02-14T18:18:33.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:18:33 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:18:33.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:33 smithi100 ceph-mon[30138]: pgmap v1690: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 908 B/s rd, 21 KiB/s wr, 2 op/s 2024-02-14T18:18:33.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:33 smithi100 ceph-mon[39557]: pgmap v1690: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 908 B/s rd, 21 KiB/s wr, 2 op/s 2024-02-14T18:18:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:34 smithi179 ceph-mon[33633]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:34 smithi179 ceph-mon[33633]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' is full (reached quota's max_objects: 10) 2024-02-14T18:18:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:34 smithi179 ceph-mon[33633]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:18:34.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:34 smithi179 ceph-mon[33633]: osdmap e773: 8 total, 8 up, 8 in 2024-02-14T18:18:34.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:34 smithi100 ceph-mon[30138]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:34 smithi100 ceph-mon[30138]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' is full (reached quota's max_objects: 10) 2024-02-14T18:18:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:34 smithi100 ceph-mon[30138]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:18:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:34 smithi100 ceph-mon[30138]: osdmap e773: 8 total, 8 up, 8 in 2024-02-14T18:18:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:34 smithi100 ceph-mon[39557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:18:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:34 smithi100 ceph-mon[39557]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' is full (reached quota's max_objects: 10) 2024-02-14T18:18:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:34 smithi100 ceph-mon[39557]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:18:34.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:34 smithi100 ceph-mon[39557]: osdmap e773: 8 total, 8 up, 8 in 2024-02-14T18:18:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:35 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:35.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:35 smithi179 ceph-mon[33633]: pgmap v1692: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-02-14T18:18:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:35 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:35.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:35 smithi100 ceph-mon[30138]: pgmap v1692: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-02-14T18:18:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:35 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:35.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:35 smithi100 ceph-mon[39557]: pgmap v1692: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-02-14T18:18:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:37 smithi179 ceph-mon[33633]: pgmap v1693: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 671 B/s rd, 16 KiB/s wr, 1 op/s 2024-02-14T18:18:37.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:37 smithi100 ceph-mon[39557]: pgmap v1693: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 671 B/s rd, 16 KiB/s wr, 1 op/s 2024-02-14T18:18:37.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:37 smithi100 ceph-mon[30138]: pgmap v1693: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 671 B/s rd, 16 KiB/s wr, 1 op/s 2024-02-14T18:18:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:38 smithi179 ceph-mon[33633]: pgmap v1694: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-14T18:18:38.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:38 smithi100 ceph-mon[30138]: pgmap v1694: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-14T18:18:38.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:38 smithi100 ceph-mon[39557]: pgmap v1694: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-14T18:18:40.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:18:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:18:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:18:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:41 smithi179 ceph-mon[33633]: pgmap v1695: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:18:41.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:41 smithi100 ceph-mon[39557]: pgmap v1695: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:18:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:41 smithi100 ceph-mon[30138]: pgmap v1695: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:18:43.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:18:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:18:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:43 smithi179 ceph-mon[33633]: pgmap v1696: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:18:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:43 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:18:43.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:43.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:43 smithi100 ceph-mon[39557]: pgmap v1696: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:18:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:43 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:18:43.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:43 smithi100 ceph-mon[30138]: pgmap v1696: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:18:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:43 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:18:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:44.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:44.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:44 smithi100 ceph-mon[39557]: pgmap v1697: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 971 B/s rd, 0 op/s 2024-02-14T18:18:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:44 smithi100 ceph-mon[30138]: pgmap v1697: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 971 B/s rd, 0 op/s 2024-02-14T18:18:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:44 smithi179 ceph-mon[33633]: pgmap v1697: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 971 B/s rd, 0 op/s 2024-02-14T18:18:47.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:47 smithi179 ceph-mon[33633]: pgmap v1698: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:47 smithi100 ceph-mon[39557]: pgmap v1698: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:47 smithi100 ceph-mon[30138]: pgmap v1698: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:49.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:49 smithi179 ceph-mon[33633]: pgmap v1699: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:49.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:49 smithi100 ceph-mon[39557]: pgmap v1699: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:49 smithi100 ceph-mon[30138]: pgmap v1699: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:50.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:18:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:18:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:18:51.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:51 smithi179 ceph-mon[33633]: pgmap v1700: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:51.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:51 smithi100 ceph-mon[30138]: pgmap v1700: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:51.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:51 smithi100 ceph-mon[39557]: pgmap v1700: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:52.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:52 smithi179 ceph-mon[33633]: pgmap v1701: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:52.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:52 smithi100 ceph-mon[30138]: pgmap v1701: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:52 smithi100 ceph-mon[39557]: pgmap v1701: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:53.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:18:53 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:18:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:55 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:55.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:55 smithi179 ceph-mon[33633]: pgmap v1702: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:55.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:55 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:55.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:55 smithi100 ceph-mon[30138]: pgmap v1702: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:55 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:18:55.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:55 smithi100 ceph-mon[39557]: pgmap v1702: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:57.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:57 smithi179 ceph-mon[33633]: pgmap v1703: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:57.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:57 smithi100 ceph-mon[30138]: pgmap v1703: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:57.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:57 smithi100 ceph-mon[39557]: pgmap v1703: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:18:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:58 smithi179 ceph-mon[33633]: pgmap v1704: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:18:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:18:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:58 smithi100 ceph-mon[30138]: pgmap v1704: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:18:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:18:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:18:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:58 smithi100 ceph-mon[39557]: pgmap v1704: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:18:58.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:18:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:18:59.990 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=156696 2024-02-14T18:18:59.991 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 8892ea91-15b7-4906-9324-dc08ab1d48a5 max_objects 100 2024-02-14T18:18:59.991 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put onemore /etc/passwd 2024-02-14T18:19:00.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 -- 172.21.15.100:0/1334765053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c10102d10 msgr2=0x7f1c1010b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:00.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 --2- 172.21.15.100:0/1334765053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c10102d10 0x7f1c1010b1f0 secure :-1 s=READY pgs=3090 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009900 tx=0x7f1c0402bcd0 comp rx=0 tx=0).stop 2024-02-14T18:19:00.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 -- 172.21.15.100:0/1334765053 shutdown_connections 2024-02-14T18:19:00.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 --2- 172.21.15.100:0/1334765053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1010b7c0 0x7f1c1010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:00.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 --2- 172.21.15.100:0/1334765053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c10102d10 0x7f1c1010b1f0 unknown :-1 s=CLOSED pgs=3090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:00.077 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 --2- 172.21.15.100:0/1334765053 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c101023f0 0x7f1c101027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 -- 172.21.15.100:0/1334765053 >> 172.21.15.100:0/1334765053 conn(0x7f1c10077260 msgr2=0x7f1c10077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 -- 172.21.15.100:0/1334765053 shutdown_connections 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 -- 172.21.15.100:0/1334765053 wait complete. 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 Processor -- start 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.074+0000 7f1c16835640 1 -- start start 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c16835640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c101023f0 0x7f1c1019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c16835640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c10102d10 0x7f1c1019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c16835640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1010b7c0 0x7f1c10196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c16835640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c10197140 con 0x7f1c101023f0 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c16835640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c101972b0 con 0x7f1c1010b7c0 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c16835640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c10197470 con 0x7f1c10102d10 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c10102d10 0x7f1c1019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c101023f0 0x7f1c1019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:00.078 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0f7fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c10102d10 0x7f1c1019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:54070/0 (socket says 172.21.15.100:54070) 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c101023f0 0x7f1c1019c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:35178/0 (socket says 172.21.15.100:35178) 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0f7fe640 1 -- 172.21.15.100:0/3564650280 learned_addr learned my addr 172.21.15.100:0/3564650280 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c14dd2640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1010b7c0 0x7f1c10196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0f7fe640 1 -- 172.21.15.100:0/3564650280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1010b7c0 msgr2=0x7f1c10196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0f7fe640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1010b7c0 0x7f1c10196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0f7fe640 1 -- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c101023f0 msgr2=0x7f1c1019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0f7fe640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c101023f0 0x7f1c1019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0f7fe640 1 -- 172.21.15.100:0/3564650280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c00009490 con 0x7f1c10102d10 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0f7fe640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c10102d10 0x7f1c1019ce80 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f1c04006010 tx=0x7f1c040042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c040044e0 con 0x7f1c10102d10 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c04009560 con 0x7f1c10102d10 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1c04003c80 con 0x7f1c10102d10 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c040054f0 con 0x7f1c10102d10 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.075+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c101a3800 con 0x7f1c10102d10 2024-02-14T18:19:00.079 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.076+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c100684b0 con 0x7f1c10102d10 2024-02-14T18:19:00.080 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.077+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f1c04003800 con 0x7f1c10102d10 2024-02-14T18:19:00.081 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.077+0000 7f1c0d7fa640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f1bf806da30 0x7f1bf806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:00.081 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.077+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(773..773 src has 254..773) v4 ==== 9047+0+0 (secure 0 0 0) 0x7f1c040b08d0 con 0x7f1c10102d10 2024-02-14T18:19:00.081 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.077+0000 7f1c0ffff640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f1bf806da30 0x7f1bf806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:00.081 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.077+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=774}) v3 -- 0x7f1bf80794a0 con 0x7f1c10102d10 2024-02-14T18:19:00.081 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.078+0000 7f1c0ffff640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f1bf806da30 0x7f1bf806fef0 secure :-1 s=READY pgs=4094 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009d60 tx=0x7f1c00005f90 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:00.082 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.082+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c04079860 con 0x7f1c10102d10 2024-02-14T18:19:00.265 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.264+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f1c10105fc0 con 0x7f1c10102d10 2024-02-14T18:19:00.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.321+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 <== mon.2 v2:172.21.15.100:3301/0 7 ==== osd_map(774..774 src has 254..774) v4 ==== 628+0+0 (secure 0 0 0) 0x7f1c04003ab0 con 0x7f1c10102d10 2024-02-14T18:19:00.322 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.321+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=775}) v3 -- 0x7f1bf8079fd0 con 0x7f1c10102d10 2024-02-14T18:19:00.327 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.326+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v774) v1 ==== 225+0+0 (secure 0 0 0) 0x7f1c04043030 con 0x7f1c10102d10 2024-02-14T18:19:00.435 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:00.434+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f1c10105fc0 con 0x7f1c10102d10 2024-02-14T18:19:00.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:00 smithi100 ceph-mon[39557]: pgmap v1705: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:00.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:00 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3564650280' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:00.484 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:00 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:00.485 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:00 smithi100 ceph-mon[30138]: pgmap v1705: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:00.485 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:00 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3564650280' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:00.485 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:00 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:00 smithi179 ceph-mon[33633]: pgmap v1705: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:00 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3564650280' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:00 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:19:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:19:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:19:01.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.327+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 <== mon.2 v2:172.21.15.100:3301/0 9 ==== osd_map(775..775 src has 254..775) v4 ==== 628+0+0 (secure 0 0 0) 0x7f1c04081660 con 0x7f1c10102d10 2024-02-14T18:19:01.328 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.327+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=776}) v3 -- 0x7f1bf807a3c0 con 0x7f1c10102d10 2024-02-14T18:19:01.332 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.331+0000 7f1c0d7fa640 1 -- 172.21.15.100:0/3564650280 <== mon.2 v2:172.21.15.100:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v775) v1 ==== 225+0+0 (secure 0 0 0) 0x7f1c04043070 con 0x7f1c10102d10 2024-02-14T18:19:01.332 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_objects = 100 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 2024-02-14T18:19:01.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f1bf806da30 msgr2=0x7f1bf806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:01.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f1bf806da30 0x7f1bf806fef0 secure :-1 s=READY pgs=4094 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009d60 tx=0x7f1c00005f90 comp rx=0 tx=0).stop 2024-02-14T18:19:01.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c10102d10 msgr2=0x7f1c1019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:01.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c10102d10 0x7f1c1019ce80 secure :-1 s=READY pgs=3132 cs=0 l=1 rev1=1 crypto rx=0x7f1c04006010 tx=0x7f1c040042f0 comp rx=0 tx=0).stop 2024-02-14T18:19:01.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 shutdown_connections 2024-02-14T18:19:01.335 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f1bf806da30 0x7f1bf806fef0 unknown :-1 s=CLOSED pgs=4094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:01.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c1010b7c0 0x7f1c10196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:01.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1c10102d10 0x7f1c1019ce80 unknown :-1 s=CLOSED pgs=3132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:01.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 --2- 172.21.15.100:0/3564650280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1c101023f0 0x7f1c1019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:01.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 >> 172.21.15.100:0/3564650280 conn(0x7f1c10077260 msgr2=0x7f1c10103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:01.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 shutdown_connections 2024-02-14T18:19:01.336 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:01.334+0000 7f1c16835640 1 -- 172.21.15.100:0/3564650280 wait complete. 2024-02-14T18:19:01.344 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 156696 2024-02-14T18:19:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]': finished 2024-02-14T18:19:01.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:01 smithi100 ceph-mon[30138]: osdmap e774: 8 total, 8 up, 8 in 2024-02-14T18:19:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:01 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3564650280' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:01 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]': finished 2024-02-14T18:19:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:01 smithi100 ceph-mon[39557]: osdmap e774: 8 total, 8 up, 8 in 2024-02-14T18:19:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:01 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3564650280' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:01 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]': finished 2024-02-14T18:19:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:01 smithi179 ceph-mon[33633]: osdmap e774: 8 total, 8 up, 8 in 2024-02-14T18:19:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:01 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3564650280' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:01.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:01 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]: dispatch 2024-02-14T18:19:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:02 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]': finished 2024-02-14T18:19:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:02 smithi179 ceph-mon[33633]: osdmap e775: 8 total, 8 up, 8 in 2024-02-14T18:19:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:02 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:02 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:02.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:02 smithi179 ceph-mon[33633]: pgmap v1708: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:02.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:02 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]': finished 2024-02-14T18:19:02.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:02 smithi100 ceph-mon[30138]: osdmap e775: 8 total, 8 up, 8 in 2024-02-14T18:19:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:02 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:02 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:02.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:02 smithi100 ceph-mon[30138]: pgmap v1708: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:02 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "100"}]': finished 2024-02-14T18:19:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:02 smithi100 ceph-mon[39557]: osdmap e775: 8 total, 8 up, 8 in 2024-02-14T18:19:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:02 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:02 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:02 smithi100 ceph-mon[39557]: pgmap v1708: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:03.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:19:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:19:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:03 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:03 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:03 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:19:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:03 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:19:03.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:03 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:03.720 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 0 -ne 0 ']' 2024-02-14T18:19:03.720 INFO:tasks.workunit.client.0.smithi100.stderr:+ true 2024-02-14T18:19:03.720 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put twomore /etc/passwd 2024-02-14T18:19:03.757 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 8892ea91-15b7-4906-9324-dc08ab1d48a5 max_bytes 100 2024-02-14T18:19:03.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 -- 172.21.15.100:0/2249471799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19701002b0 msgr2=0x7f1970100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:03.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 --2- 172.21.15.100:0/2249471799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19701002b0 0x7f1970100690 secure :-1 s=READY pgs=3135 cs=0 l=1 rev1=1 crypto rx=0x7f1964009860 tx=0x7f196402bb70 comp rx=0 tx=0).stop 2024-02-14T18:19:03.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 -- 172.21.15.100:0/2249471799 shutdown_connections 2024-02-14T18:19:03.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 --2- 172.21.15.100:0/2249471799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1970105380 0x7f197010cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:03.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 --2- 172.21.15.100:0/2249471799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1970100c60 0x7f1970104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:03.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 --2- 172.21.15.100:0/2249471799 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19701002b0 0x7f1970100690 unknown :-1 s=CLOSED pgs=3135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:03.844 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 -- 172.21.15.100:0/2249471799 >> 172.21.15.100:0/2249471799 conn(0x7f19700771a0 msgr2=0x7f19700775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 -- 172.21.15.100:0/2249471799 shutdown_connections 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 -- 172.21.15.100:0/2249471799 wait complete. 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 Processor -- start 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.841+0000 7f1976a96640 1 -- start start 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1976a96640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19701002b0 0x7f1970198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1976a96640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1970105380 0x7f1970199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1976a96640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197019d8a0 0x7f197019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1976a96640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197019e290 con 0x7f197019d8a0 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1976a96640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197019e400 con 0x7f1970105380 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1976a96640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197019e570 con 0x7f19701002b0 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1974832640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19701002b0 0x7f1970198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1975033640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197019d8a0 0x7f197019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1974832640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19701002b0 0x7f1970198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:34204/0 (socket says 172.21.15.100:34204) 2024-02-14T18:19:03.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1974832640 1 -- 172.21.15.100:0/4119741413 learned_addr learned my addr 172.21.15.100:0/4119741413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f196ffff640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1970105380 0x7f1970199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1975033640 1 -- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19701002b0 msgr2=0x7f1970198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1975033640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19701002b0 0x7f1970198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1975033640 1 -- 172.21.15.100:0/4119741413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1970105380 msgr2=0x7f1970199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1975033640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1970105380 0x7f1970199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1975033640 1 -- 172.21.15.100:0/4119741413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968009900 con 0x7f197019d8a0 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f196ffff640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1970105380 0x7f1970199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1974832640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19701002b0 0x7f1970198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1975033640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197019d8a0 0x7f197019dd50 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7f196800efc0 tx=0x7f196800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f196dffb640 1 -- 172.21.15.100:0/4119741413 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1968003970 con 0x7f197019d8a0 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1964009490 con 0x7f197019d8a0 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f196dffb640 1 -- 172.21.15.100:0/4119741413 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1968004020 con 0x7f197019d8a0 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f196dffb640 1 -- 172.21.15.100:0/4119741413 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800b870 con 0x7f197019d8a0 2024-02-14T18:19:03.846 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.842+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19701a5e70 con 0x7f197019d8a0 2024-02-14T18:19:03.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.843+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1970068060 con 0x7f197019d8a0 2024-02-14T18:19:03.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.844+0000 7f196dffb640 1 -- 172.21.15.100:0/4119741413 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f196800a040 con 0x7f197019d8a0 2024-02-14T18:19:03.847 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.844+0000 7f196dffb640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f196006da30 0x7f196006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:03.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.844+0000 7f196dffb640 1 -- 172.21.15.100:0/4119741413 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(776..776 src has 254..776) v4 ==== 9047+0+0 (secure 0 0 0) 0x7f196808e0c0 con 0x7f197019d8a0 2024-02-14T18:19:03.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.844+0000 7f1974832640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f196006da30 0x7f196006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:03.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.845+0000 7f1974832640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f196006da30 0x7f196006fef0 secure :-1 s=READY pgs=4096 cs=0 l=1 rev1=1 crypto rx=0x7f19640057c0 tx=0x7f196402e040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:03.849 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:03.849+0000 7f196dffb640 1 -- 172.21.15.100:0/4119741413 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1968056e80 con 0x7f197019d8a0 2024-02-14T18:19:03.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:03 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:03.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:03 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:03.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:03 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:19:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:03 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:19:03.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:03 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:03 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:03 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:03 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:19:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:03 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:19:03.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:03 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:04.024 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:04.024+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f1970100c60 con 0x7f197019d8a0 2024-02-14T18:19:04.713 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:04.712+0000 7f196dffb640 1 -- 172.21.15.100:0/4119741413 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v777) v1 ==== 221+0+0 (secure 0 0 0) 0x7f196805a540 con 0x7f197019d8a0 2024-02-14T18:19:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:04 smithi179 ceph-mon[33633]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' no longer out of quota; removing NO_QUOTA flag 2024-02-14T18:19:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:04 smithi179 ceph-mon[33633]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:19:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:04 smithi179 ceph-mon[33633]: osdmap e776: 8 total, 8 up, 8 in 2024-02-14T18:19:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:04 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T18:19:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:04 smithi179 ceph-mon[33633]: pgmap v1710: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:04.826 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:04.826+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f197010f690 con 0x7f197019d8a0 2024-02-14T18:19:04.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[39557]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' no longer out of quota; removing NO_QUOTA flag 2024-02-14T18:19:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[39557]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:19:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[39557]: osdmap e776: 8 total, 8 up, 8 in 2024-02-14T18:19:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T18:19:04.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[39557]: pgmap v1710: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[30138]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' no longer out of quota; removing NO_QUOTA flag 2024-02-14T18:19:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[30138]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:19:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[30138]: osdmap e776: 8 total, 8 up, 8 in 2024-02-14T18:19:04.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T18:19:04.901 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:04 smithi100 ceph-mon[30138]: pgmap v1710: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:05.715 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.714+0000 7f196dffb640 1 -- 172.21.15.100:0/4119741413 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v778) v1 ==== 221+0+0 (secure 0 0 0) 0x7f196805a720 con 0x7f197019d8a0 2024-02-14T18:19:05.715 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_bytes = 100 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 2024-02-14T18:19:05.720 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f196006da30 msgr2=0x7f196006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:05.720 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f196006da30 0x7f196006fef0 secure :-1 s=READY pgs=4096 cs=0 l=1 rev1=1 crypto rx=0x7f19640057c0 tx=0x7f196402e040 comp rx=0 tx=0).stop 2024-02-14T18:19:05.720 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197019d8a0 msgr2=0x7f197019dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:05.720 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197019d8a0 0x7f197019dd50 secure :-1 s=READY pgs=3092 cs=0 l=1 rev1=1 crypto rx=0x7f196800efc0 tx=0x7f196800c800 comp rx=0 tx=0).stop 2024-02-14T18:19:05.720 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 shutdown_connections 2024-02-14T18:19:05.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f196006da30 0x7f196006fef0 unknown :-1 s=CLOSED pgs=4096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:05.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f197019d8a0 0x7f197019dd50 unknown :-1 s=CLOSED pgs=3092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:05.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1970105380 0x7f1970199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:05.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 --2- 172.21.15.100:0/4119741413 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f19701002b0 0x7f1970198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:05.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 >> 172.21.15.100:0/4119741413 conn(0x7f19700771a0 msgr2=0x7f19700684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:05.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 shutdown_connections 2024-02-14T18:19:05.721 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:05.719+0000 7f1976a96640 1 -- 172.21.15.100:0/4119741413 wait complete. 2024-02-14T18:19:05.733 INFO:tasks.workunit.client.0.smithi100.stderr:+ sleep 30 2024-02-14T18:19:06.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T18:19:06.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:05 smithi179 ceph-mon[33633]: osdmap e777: 8 total, 8 up, 8 in 2024-02-14T18:19:06.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:05 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T18:19:06.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T18:19:06.149 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:05 smithi100 ceph-mon[39557]: osdmap e777: 8 total, 8 up, 8 in 2024-02-14T18:19:06.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:05 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T18:19:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T18:19:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:05 smithi100 ceph-mon[30138]: osdmap e777: 8 total, 8 up, 8 in 2024-02-14T18:19:06.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:05 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-14T18:19:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:06 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T18:19:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:06 smithi179 ceph-mon[33633]: osdmap e778: 8 total, 8 up, 8 in 2024-02-14T18:19:07.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:06 smithi179 ceph-mon[33633]: pgmap v1713: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:19:07.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:06 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T18:19:07.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:06 smithi100 ceph-mon[30138]: osdmap e778: 8 total, 8 up, 8 in 2024-02-14T18:19:07.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:06 smithi100 ceph-mon[30138]: pgmap v1713: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:19:07.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:06 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/4119741413' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "100"}]': finished 2024-02-14T18:19:07.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:06 smithi100 ceph-mon[39557]: osdmap e778: 8 total, 8 up, 8 in 2024-02-14T18:19:07.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:06 smithi100 ceph-mon[39557]: pgmap v1713: 176 pgs: 176 active+clean; 697 KiB data, 886 MiB used, 714 GiB / 715 GiB avail 2024-02-14T18:19:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:09 smithi179 ceph-mon[33633]: pgmap v1714: 176 pgs: 176 active+clean; 720 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:19:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:09 smithi179 ceph-mon[33633]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' is full (reached quota's max_bytes: 100 B) 2024-02-14T18:19:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:09 smithi179 ceph-mon[33633]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:19:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:09 smithi179 ceph-mon[33633]: osdmap e779: 8 total, 8 up, 8 in 2024-02-14T18:19:09.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:09 smithi100 ceph-mon[39557]: pgmap v1714: 176 pgs: 176 active+clean; 720 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:19:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:09 smithi100 ceph-mon[39557]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' is full (reached quota's max_bytes: 100 B) 2024-02-14T18:19:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:09 smithi100 ceph-mon[39557]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:19:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:09 smithi100 ceph-mon[39557]: osdmap e779: 8 total, 8 up, 8 in 2024-02-14T18:19:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:09 smithi100 ceph-mon[30138]: pgmap v1714: 176 pgs: 176 active+clean; 720 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:19:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:09 smithi100 ceph-mon[30138]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' is full (reached quota's max_bytes: 100 B) 2024-02-14T18:19:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:09 smithi100 ceph-mon[30138]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:19:09.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:09 smithi100 ceph-mon[30138]: osdmap e779: 8 total, 8 up, 8 in 2024-02-14T18:19:10.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:10 smithi100 ceph-mon[30138]: pgmap v1716: 176 pgs: 176 active+clean; 720 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:19:10.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:19:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:19:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:19:10.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:10 smithi100 ceph-mon[39557]: pgmap v1716: 176 pgs: 176 active+clean; 720 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:19:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:10 smithi179 ceph-mon[33633]: pgmap v1716: 176 pgs: 176 active+clean; 720 KiB data, 886 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:19:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:13 smithi179 ceph-mon[33633]: pgmap v1717: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-14T18:19:13.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:13.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:19:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:19:13.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:13 smithi100 ceph-mon[39557]: pgmap v1717: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-14T18:19:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:13 smithi100 ceph-mon[30138]: pgmap v1717: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-14T18:19:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:14.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:14 smithi179 ceph-mon[33633]: pgmap v1718: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-02-14T18:19:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:14.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:14 smithi100 ceph-mon[30138]: pgmap v1718: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-02-14T18:19:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:14.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:14 smithi100 ceph-mon[39557]: pgmap v1718: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.8 KiB/s wr, 1 op/s 2024-02-14T18:19:17.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:17 smithi179 ceph-mon[33633]: pgmap v1719: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-14T18:19:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:17 smithi100 ceph-mon[30138]: pgmap v1719: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-14T18:19:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:17 smithi100 ceph-mon[39557]: pgmap v1719: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-14T18:19:18.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:18 smithi179 ceph-mon[33633]: pgmap v1720: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:19:18.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:18 smithi100 ceph-mon[30138]: pgmap v1720: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:19:18.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:18 smithi100 ceph-mon[39557]: pgmap v1720: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:19:20.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:20 smithi100 ceph-mon[39557]: pgmap v1721: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 887 B/s rd, 0 op/s 2024-02-14T18:19:20.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:20 smithi100 ceph-mon[30138]: pgmap v1721: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 887 B/s rd, 0 op/s 2024-02-14T18:19:20.651 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:19:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:19:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:19:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:20 smithi179 ceph-mon[33633]: pgmap v1721: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 887 B/s rd, 0 op/s 2024-02-14T18:19:22.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:22 smithi100 ceph-mon[30138]: pgmap v1722: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:22.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:22 smithi100 ceph-mon[39557]: pgmap v1722: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:22.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:22 smithi179 ceph-mon[33633]: pgmap v1722: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:23.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:19:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:19:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:24 smithi100 ceph-mon[30138]: pgmap v1723: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:24 smithi100 ceph-mon[39557]: pgmap v1723: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:24 smithi179 ceph-mon[33633]: pgmap v1723: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:26.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:26 smithi100 ceph-mon[30138]: pgmap v1724: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:26.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:26 smithi100 ceph-mon[39557]: pgmap v1724: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:26 smithi179 ceph-mon[33633]: pgmap v1724: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:28.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:28 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:28.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:28 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:28.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:28 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:29 smithi179 ceph-mon[33633]: pgmap v1725: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:29.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:29 smithi100 ceph-mon[30138]: pgmap v1725: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:29 smithi100 ceph-mon[39557]: pgmap v1725: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:30.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:30 smithi100 ceph-mon[39557]: pgmap v1726: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:30.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:30 smithi100 ceph-mon[30138]: pgmap v1726: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:30.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:19:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:19:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:19:30.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:30 smithi179 ceph-mon[33633]: pgmap v1726: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:32.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:32 smithi100 ceph-mon[39557]: pgmap v1727: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:32.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:32 smithi100 ceph-mon[30138]: pgmap v1727: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:32.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:32 smithi179 ceph-mon[33633]: pgmap v1727: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:33.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:19:33 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:19:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:34 smithi100 ceph-mon[39557]: pgmap v1728: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:34 smithi100 ceph-mon[30138]: pgmap v1728: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:34 smithi179 ceph-mon[33633]: pgmap v1728: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:35.735 INFO:tasks.workunit.client.0.smithi100.stderr:+ pid=157051 2024-02-14T18:19:35.735 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 8892ea91-15b7-4906-9324-dc08ab1d48a5 max_bytes 0 2024-02-14T18:19:35.735 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put two /etc/passwd 2024-02-14T18:19:35.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.830+0000 7f72a9520640 1 -- 172.21.15.100:0/2950466973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72a41023f0 msgr2=0x7f72a41027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:35.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.830+0000 7f72a9520640 1 --2- 172.21.15.100:0/2950466973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72a41023f0 0x7f72a41027d0 secure :-1 s=READY pgs=3136 cs=0 l=1 rev1=1 crypto rx=0x7f7298009900 tx=0x7f729802bcd0 comp rx=0 tx=0).stop 2024-02-14T18:19:35.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.830+0000 7f72a9520640 1 -- 172.21.15.100:0/2950466973 shutdown_connections 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.830+0000 7f72a9520640 1 --2- 172.21.15.100:0/2950466973 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a410b7c0 0x7f72a410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.830+0000 7f72a9520640 1 --2- 172.21.15.100:0/2950466973 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a4102d10 0x7f72a410b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.830+0000 7f72a9520640 1 --2- 172.21.15.100:0/2950466973 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72a41023f0 0x7f72a41027d0 unknown :-1 s=CLOSED pgs=3136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.830+0000 7f72a9520640 1 -- 172.21.15.100:0/2950466973 >> 172.21.15.100:0/2950466973 conn(0x7f72a4077170 msgr2=0x7f72a4077580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.830+0000 7f72a9520640 1 -- 172.21.15.100:0/2950466973 shutdown_connections 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.830+0000 7f72a9520640 1 -- 172.21.15.100:0/2950466973 wait complete. 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.830+0000 7f72a9520640 1 Processor -- start 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a9520640 1 -- start start 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a9520640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a41023f0 0x7f72a4196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a9520640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72a4102d10 0x7f72a4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a9520640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a410b7c0 0x7f72a419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a9520640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419bcb0 con 0x7f72a410b7c0 2024-02-14T18:19:35.834 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a9520640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419be20 con 0x7f72a41023f0 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a9520640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419bfe0 con 0x7f72a4102d10 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a27fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72a4102d10 0x7f72a4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a410b7c0 0x7f72a419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a27fc640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72a4102d10 0x7f72a4197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:43944/0 (socket says 172.21.15.100:43944) 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a27fc640 1 -- 172.21.15.100:0/2328842618 learned_addr learned my addr 172.21.15.100:0/2328842618 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a2ffd640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a41023f0 0x7f72a4196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a27fc640 1 -- 172.21.15.100:0/2328842618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a41023f0 msgr2=0x7f72a4196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a27fc640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a41023f0 0x7f72a4196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a27fc640 1 -- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a410b7c0 msgr2=0x7f72a419b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a27fc640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a410b7c0 0x7f72a419b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a27fc640 1 -- 172.21.15.100:0/2328842618 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294009490 con 0x7f72a4102d10 2024-02-14T18:19:35.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a2ffd640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a41023f0 0x7f72a4196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T18:19:35.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a37fe640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a410b7c0 0x7f72a419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:19:35.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a27fc640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72a4102d10 0x7f72a4197330 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 crypto rx=0x7f729400ea60 tx=0x7f729400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:35.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729400cde0 con 0x7f72a4102d10 2024-02-14T18:19:35.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7298009560 con 0x7f72a4102d10 2024-02-14T18:19:35.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f72940044a0 con 0x7f72a4102d10 2024-02-14T18:19:35.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.831+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294010430 con 0x7f72a4102d10 2024-02-14T18:19:35.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.832+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a41a3960 con 0x7f72a4102d10 2024-02-14T18:19:35.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.832+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a40684b0 con 0x7f72a4102d10 2024-02-14T18:19:35.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.833+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f7294003f60 con 0x7f72a4102d10 2024-02-14T18:19:35.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.833+0000 7f728bfff640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f728c06da30 0x7f728c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:35.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.834+0000 7f72a2ffd640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f728c06da30 0x7f728c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:35.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.834+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(779..779 src has 254..779) v4 ==== 9047+0+0 (secure 0 0 0) 0x7f7294014070 con 0x7f72a4102d10 2024-02-14T18:19:35.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.834+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=780}) v3 -- 0x7f728c0794a0 con 0x7f72a4102d10 2024-02-14T18:19:35.837 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.834+0000 7f72a2ffd640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f728c06da30 0x7f728c06fef0 secure :-1 s=READY pgs=4098 cs=0 l=1 rev1=1 crypto rx=0x7f7298036040 tx=0x7f7298000bc0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:35.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:35.838+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f72940571d0 con 0x7f72a4102d10 2024-02-14T18:19:36.015 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:36.014+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f72a4110720 con 0x7f72a4102d10 2024-02-14T18:19:36.076 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:36.075+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 <== mon.2 v2:172.21.15.100:3301/0 7 ==== osd_map(780..780 src has 254..780) v4 ==== 628+0+0 (secure 0 0 0) 0x7f729405b630 con 0x7f72a4102d10 2024-02-14T18:19:36.076 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:36.075+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=781}) v3 -- 0x7f728c079fd0 con 0x7f72a4102d10 2024-02-14T18:19:36.080 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:36.079+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v780) v1 ==== 217+0+0 (secure 0 0 0) 0x7f729405a890 con 0x7f72a4102d10 2024-02-14T18:19:36.187 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:36.186+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f72a4110930 con 0x7f72a4102d10 2024-02-14T18:19:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:36 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2328842618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:36.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:36 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:36.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:36 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2328842618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:36.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:36 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:36.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:36 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2328842618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:36.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:36 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:37.082 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.081+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 <== mon.2 v2:172.21.15.100:3301/0 9 ==== osd_map(781..781 src has 254..781) v4 ==== 628+0+0 (secure 0 0 0) 0x7f729405a890 con 0x7f72a4102d10 2024-02-14T18:19:37.082 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.081+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=782}) v3 -- 0x7f728c07a400 con 0x7f72a4102d10 2024-02-14T18:19:37.087 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.086+0000 7f728bfff640 1 -- 172.21.15.100:0/2328842618 <== mon.2 v2:172.21.15.100:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v781) v1 ==== 217+0+0 (secure 0 0 0) 0x7f7294072030 con 0x7f72a4102d10 2024-02-14T18:19:37.087 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_bytes = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 2024-02-14T18:19:37.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f728c06da30 msgr2=0x7f728c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:37.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f728c06da30 0x7f728c06fef0 secure :-1 s=READY pgs=4098 cs=0 l=1 rev1=1 crypto rx=0x7f7298036040 tx=0x7f7298000bc0 comp rx=0 tx=0).stop 2024-02-14T18:19:37.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72a4102d10 msgr2=0x7f72a4197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:37.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72a4102d10 0x7f72a4197330 secure :-1 s=READY pgs=3137 cs=0 l=1 rev1=1 crypto rx=0x7f729400ea60 tx=0x7f729400ef20 comp rx=0 tx=0).stop 2024-02-14T18:19:37.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 shutdown_connections 2024-02-14T18:19:37.091 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f728c06da30 0x7f728c06fef0 unknown :-1 s=CLOSED pgs=4098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:37.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72a410b7c0 0x7f72a419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:37.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72a4102d10 0x7f72a4197330 unknown :-1 s=CLOSED pgs=3137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:37.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 --2- 172.21.15.100:0/2328842618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72a41023f0 0x7f72a4196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:37.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 >> 172.21.15.100:0/2328842618 conn(0x7f72a4077170 msgr2=0x7f72a40fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:37.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 shutdown_connections 2024-02-14T18:19:37.092 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.091+0000 7f72a9520640 1 -- 172.21.15.100:0/2328842618 wait complete. 2024-02-14T18:19:37.102 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 8892ea91-15b7-4906-9324-dc08ab1d48a5 max_objects 0 2024-02-14T18:19:37.188 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.188+0000 7ff8a9d10640 1 -- 172.21.15.100:0/1720454354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a4101770 msgr2=0x7ff8a4104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:37.188 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.188+0000 7ff8a9d10640 1 --2- 172.21.15.100:0/1720454354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a4101770 0x7ff8a4104d10 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7ff898009890 tx=0x7ff89802bb70 comp rx=0 tx=0).stop 2024-02-14T18:19:37.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.188+0000 7ff8a9d10640 1 -- 172.21.15.100:0/1720454354 shutdown_connections 2024-02-14T18:19:37.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.188+0000 7ff8a9d10640 1 --2- 172.21.15.100:0/1720454354 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8a4105440 0x7ff8a410cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:37.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.188+0000 7ff8a9d10640 1 --2- 172.21.15.100:0/1720454354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a4101770 0x7ff8a4104d10 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:37.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.188+0000 7ff8a9d10640 1 --2- 172.21.15.100:0/1720454354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8a4100dc0 0x7ff8a41011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:37.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.188+0000 7ff8a9d10640 1 -- 172.21.15.100:0/1720454354 >> 172.21.15.100:0/1720454354 conn(0x7ff8a40753a0 msgr2=0x7ff8a40757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:37.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.188+0000 7ff8a9d10640 1 -- 172.21.15.100:0/1720454354 shutdown_connections 2024-02-14T18:19:37.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.188+0000 7ff8a9d10640 1 -- 172.21.15.100:0/1720454354 wait complete. 2024-02-14T18:19:37.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.188+0000 7ff8a9d10640 1 Processor -- start 2024-02-14T18:19:37.189 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a9d10640 1 -- start start 2024-02-14T18:19:37.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a9d10640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a4100dc0 0x7ff8a4196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:37.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a9d10640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8a4105440 0x7ff8a41972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:37.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a9d10640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8a419b620 0x7ff8a419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:37.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a9d10640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a419c010 con 0x7ff8a4100dc0 2024-02-14T18:19:37.192 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a9d10640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a419c180 con 0x7ff8a419b620 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a9d10640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a419c2f0 con 0x7ff8a4105440 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8a4105440 0x7ff8a41972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a4100dc0 0x7ff8a4196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a2ffd640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8a4105440 0x7ff8a41972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:43968/0 (socket says 172.21.15.100:43968) 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a4100dc0 0x7ff8a4196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39838/0 (socket says 172.21.15.100:39838) 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a2ffd640 1 -- 172.21.15.100:0/3898921720 learned_addr learned my addr 172.21.15.100:0/3898921720 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a3fff640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8a419b620 0x7ff8a419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a2ffd640 1 -- 172.21.15.100:0/3898921720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8a419b620 msgr2=0x7ff8a419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a2ffd640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8a419b620 0x7ff8a419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a2ffd640 1 -- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a4100dc0 msgr2=0x7ff8a4196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:37.193 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a2ffd640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a4100dc0 0x7ff8a4196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a2ffd640 1 -- 172.21.15.100:0/3898921720 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c009900 con 0x7ff8a4105440 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a3fff640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8a419b620 0x7ff8a419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a37fe640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a4100dc0 0x7ff8a4196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.189+0000 7ff8a2ffd640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8a4105440 0x7ff8a41972a0 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7ff89802c180 tx=0x7ff898004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.190+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89802c510 con 0x7ff8a4105440 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.190+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff898009490 con 0x7ff8a4105440 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.190+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff89802caf0 con 0x7ff8a4105440 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.190+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89803d3f0 con 0x7ff8a4105440 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.190+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a41a3bc0 con 0x7ff8a4105440 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.191+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a40684b0 con 0x7ff8a4105440 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.191+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7ff89803b040 con 0x7ff8a4105440 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.192+0000 7ff8a0ff9640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff88c06da30 0x7ff88c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:37.194 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.192+0000 7ff8a37fe640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff88c06da30 0x7ff88c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:37.195 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.192+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(781..781 src has 254..781) v4 ==== 9047+0+0 (secure 0 0 0) 0x7ff8980af970 con 0x7ff8a4105440 2024-02-14T18:19:37.195 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.192+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=782}) v3 -- 0x7ff88c0794a0 con 0x7ff8a4105440 2024-02-14T18:19:37.195 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.192+0000 7ff8a37fe640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff88c06da30 0x7ff88c06fef0 secure :-1 s=READY pgs=4099 cs=0 l=1 rev1=1 crypto rx=0x7ff8940058c0 tx=0x7ff89400a2d0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:37.196 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.196+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff8980787e0 con 0x7ff8a4105440 2024-02-14T18:19:37.370 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:37.369+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7ff8a4108640 con 0x7ff8a4105440 2024-02-14T18:19:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:19:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:37 smithi100 ceph-mon[30138]: osdmap e780: 8 total, 8 up, 8 in 2024-02-14T18:19:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:37 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2328842618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:37.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:37 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:37.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:37 smithi100 ceph-mon[30138]: pgmap v1730: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:19:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:19:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:37 smithi100 ceph-mon[39557]: osdmap e780: 8 total, 8 up, 8 in 2024-02-14T18:19:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:37 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2328842618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:37 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:37.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:37 smithi100 ceph-mon[39557]: pgmap v1730: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:19:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:19:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:37 smithi179 ceph-mon[33633]: osdmap e780: 8 total, 8 up, 8 in 2024-02-14T18:19:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:37 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2328842618' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:37 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:19:37.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:37 smithi179 ceph-mon[33633]: pgmap v1730: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:19:38.084 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.083+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 <== mon.2 v2:172.21.15.100:3301/0 7 ==== osd_map(782..782 src has 254..782) v4 ==== 628+0+0 (secure 0 0 0) 0x7ff8980805e0 con 0x7ff8a4105440 2024-02-14T18:19:38.084 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.083+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=783}) v3 -- 0x7ff88c079fd0 con 0x7ff8a4105440 2024-02-14T18:19:38.087 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.087+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v782) v1 ==== 221+0+0 (secure 0 0 0) 0x7ff898039020 con 0x7ff8a4105440 2024-02-14T18:19:38.198 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.197+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7ff8a4108640 con 0x7ff8a4105440 2024-02-14T18:19:38.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:19:38.399 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:38 smithi100 ceph-mon[30138]: osdmap e781: 8 total, 8 up, 8 in 2024-02-14T18:19:38.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:38 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3898921720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:38.400 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:38 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:38.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:19:38.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:38 smithi100 ceph-mon[39557]: osdmap e781: 8 total, 8 up, 8 in 2024-02-14T18:19:38.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:38 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3898921720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:38.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:38 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:19:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:38 smithi179 ceph-mon[33633]: osdmap e781: 8 total, 8 up, 8 in 2024-02-14T18:19:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:38 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3898921720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:38.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:38 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:38.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.717+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 <== mon.2 v2:172.21.15.100:3301/0 9 ==== osd_map(783..783 src has 254..783) v4 ==== 628+0+0 (secure 0 0 0) 0x7ff898084ba0 con 0x7ff8a4105440 2024-02-14T18:19:38.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.717+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=784}) v3 -- 0x7ff88c07a3c0 con 0x7ff8a4105440 2024-02-14T18:19:38.720 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.720+0000 7ff8a0ff9640 1 -- 172.21.15.100:0/3898921720 <== mon.2 v2:172.21.15.100:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v783) v1 ==== 221+0+0 (secure 0 0 0) 0x7ff89802ce60 con 0x7ff8a4105440 2024-02-14T18:19:38.721 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_objects = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 2024-02-14T18:19:38.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff88c06da30 msgr2=0x7ff88c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:38.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff88c06da30 0x7ff88c06fef0 secure :-1 s=READY pgs=4099 cs=0 l=1 rev1=1 crypto rx=0x7ff8940058c0 tx=0x7ff89400a2d0 comp rx=0 tx=0).stop 2024-02-14T18:19:38.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8a4105440 msgr2=0x7ff8a41972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:38.724 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8a4105440 0x7ff8a41972a0 secure :-1 s=READY pgs=3138 cs=0 l=1 rev1=1 crypto rx=0x7ff89802c180 tx=0x7ff898004040 comp rx=0 tx=0).stop 2024-02-14T18:19:38.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 shutdown_connections 2024-02-14T18:19:38.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff88c06da30 0x7ff88c06fef0 unknown :-1 s=CLOSED pgs=4099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:38.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8a419b620 0x7ff8a419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:38.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff8a4105440 0x7ff8a41972a0 unknown :-1 s=CLOSED pgs=3138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:38.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 --2- 172.21.15.100:0/3898921720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a4100dc0 0x7ff8a4196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:38.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 >> 172.21.15.100:0/3898921720 conn(0x7ff8a40753a0 msgr2=0x7ff8a40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:38.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 shutdown_connections 2024-02-14T18:19:38.725 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.723+0000 7ff8a9d10640 1 -- 172.21.15.100:0/3898921720 wait complete. 2024-02-14T18:19:38.734 INFO:tasks.workunit.client.0.smithi100.stderr:+ wait 157051 2024-02-14T18:19:38.734 INFO:tasks.workunit.client.0.smithi100.stderr:+ '[' 0 -ne 0 ']' 2024-02-14T18:19:38.735 INFO:tasks.workunit.client.0.smithi100.stderr:+ true 2024-02-14T18:19:38.735 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put three /etc/passwd 2024-02-14T18:19:38.771 INFO:tasks.workunit.client.0.smithi100.stderr:++ uuidgen 2024-02-14T18:19:38.772 INFO:tasks.workunit.client.0.smithi100.stderr:+ pp=36e0b1f1-ec9e-41a3-bde9-02e94ccf390d 2024-02-14T18:19:38.772 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool create 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d 12 2024-02-14T18:19:38.857 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 -- 172.21.15.100:0/3639523352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72681002b0 msgr2=0x7f7268100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:38.857 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 --2- 172.21.15.100:0/3639523352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72681002b0 0x7f7268100690 secure :-1 s=READY pgs=3140 cs=0 l=1 rev1=1 crypto rx=0x7f7258009860 tx=0x7f725802bb70 comp rx=0 tx=0).stop 2024-02-14T18:19:38.857 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 -- 172.21.15.100:0/3639523352 shutdown_connections 2024-02-14T18:19:38.857 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 --2- 172.21.15.100:0/3639523352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7268105380 0x7f726810cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:38.857 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 --2- 172.21.15.100:0/3639523352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7268100c60 0x7f7268104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:38.857 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 --2- 172.21.15.100:0/3639523352 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72681002b0 0x7f7268100690 unknown :-1 s=CLOSED pgs=3140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:38.857 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 -- 172.21.15.100:0/3639523352 >> 172.21.15.100:0/3639523352 conn(0x7f72680771a0 msgr2=0x7f72680775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 -- 172.21.15.100:0/3639523352 shutdown_connections 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 -- 172.21.15.100:0/3639523352 wait complete. 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 Processor -- start 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.854+0000 7f726cd77640 1 -- start start 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726cd77640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72681002b0 0x7f726810d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726cd77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7268105380 0x7f726810dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726cd77640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726810e2b0 0x7f72681a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726cd77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681a5510 con 0x7f7268105380 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726cd77640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72681a5680 con 0x7f726810e2b0 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726cd77640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72681a5840 con 0x7f72681002b0 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726659c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72681002b0 0x7f726810d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:38.858 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f7265d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7268105380 0x7f726810dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726659c640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72681002b0 0x7f726810d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:44016/0 (socket says 172.21.15.100:44016) 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726659c640 1 -- 172.21.15.100:0/1578550965 learned_addr learned my addr 172.21.15.100:0/1578550965 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f7266d9d640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726810e2b0 0x7f72681a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f7265d9b640 1 -- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72681002b0 msgr2=0x7f726810d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f7265d9b640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72681002b0 0x7f726810d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f7265d9b640 1 -- 172.21.15.100:0/1578550965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726810e2b0 msgr2=0x7f72681a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f7265d9b640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726810e2b0 0x7f72681a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f7265d9b640 1 -- 172.21.15.100:0/1578550965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7260009900 con 0x7f7268105380 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f7266d9d640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726810e2b0 0x7f72681a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726659c640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72681002b0 0x7f726810d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f7265d9b640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7268105380 0x7f726810dd70 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 crypto rx=0x7f725c00d6a0 tx=0x7f725c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f72577fe640 1 -- 172.21.15.100:0/1578550965 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c005220 con 0x7f7268105380 2024-02-14T18:19:38.859 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7258009490 con 0x7f7268105380 2024-02-14T18:19:38.860 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f72577fe640 1 -- 172.21.15.100:0/1578550965 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f725c009ca0 con 0x7f7268105380 2024-02-14T18:19:38.860 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f72577fe640 1 -- 172.21.15.100:0/1578550965 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c004b80 con 0x7f7268105380 2024-02-14T18:19:38.860 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.855+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72681a5ef0 con 0x7f7268105380 2024-02-14T18:19:38.860 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.856+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7268068060 con 0x7f7268105380 2024-02-14T18:19:38.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.857+0000 7f72577fe640 1 -- 172.21.15.100:0/1578550965 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f725c004ce0 con 0x7f7268105380 2024-02-14T18:19:38.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.857+0000 7f72577fe640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f725006da30 0x7f725006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:38.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.857+0000 7f726659c640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f725006da30 0x7f725006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:38.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.857+0000 7f72577fe640 1 -- 172.21.15.100:0/1578550965 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(783..783 src has 254..783) v4 ==== 9047+0+0 (secure 0 0 0) 0x7f725c08d7e0 con 0x7f7268105380 2024-02-14T18:19:38.861 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.858+0000 7f726659c640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f725006da30 0x7f725006fef0 secure :-1 s=READY pgs=4101 cs=0 l=1 rev1=1 crypto rx=0x7f72580057c0 tx=0x7f725802e040 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:38.863 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:38.862+0000 7f72577fe640 1 -- 172.21.15.100:0/1578550965 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f725c056650 con 0x7f7268105380 2024-02-14T18:19:39.040 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.039+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12} v 0) v1 -- 0x7f72681085b0 con 0x7f7268105380 2024-02-14T18:19:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:19:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:39 smithi179 ceph-mon[33633]: osdmap e782: 8 total, 8 up, 8 in 2024-02-14T18:19:39.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:39 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3898921720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:39.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:39.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:39 smithi179 ceph-mon[33633]: pgmap v1733: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:39.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:39 smithi179 ceph-mon[33633]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' no longer out of quota; removing NO_QUOTA flag 2024-02-14T18:19:39.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:39 smithi179 ceph-mon[33633]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:19:39.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:39 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:19:39.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:39 smithi179 ceph-mon[33633]: osdmap e783: 8 total, 8 up, 8 in 2024-02-14T18:19:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:19:39.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[30138]: osdmap e782: 8 total, 8 up, 8 in 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3898921720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[30138]: pgmap v1733: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[30138]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' no longer out of quota; removing NO_QUOTA flag 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[30138]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[30138]: osdmap e783: 8 total, 8 up, 8 in 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[39557]: osdmap e782: 8 total, 8 up, 8 in 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3898921720' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:39.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:19:39.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[39557]: pgmap v1733: 176 pgs: 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:39.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[39557]: pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' no longer out of quota; removing NO_QUOTA flag 2024-02-14T18:19:39.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[39557]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:19:39.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:19:39.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:39 smithi100 ceph-mon[39557]: osdmap e783: 8 total, 8 up, 8 in 2024-02-14T18:19:39.718 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.717+0000 7f72577fe640 1 -- 172.21.15.100:0/1578550965 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]=0 pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' created v784) v1 ==== 176+0+0 (secure 0 0 0) 0x7f725c059d10 con 0x7f7268105380 2024-02-14T18:19:39.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.829+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12} v 0) v1 -- 0x7f726810ed10 con 0x7f7268105380 2024-02-14T18:19:39.830 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.829+0000 7f72577fe640 1 -- 172.21.15.100:0/1578550965 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]=0 pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' already exists v784) v1 ==== 183+0+0 (secure 0 0 0) 0x7f725c020030 con 0x7f7268105380 2024-02-14T18:19:39.830 INFO:tasks.workunit.client.0.smithi100.stderr:pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' already exists 2024-02-14T18:19:39.832 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f725006da30 msgr2=0x7f725006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f725006da30 0x7f725006fef0 secure :-1 s=READY pgs=4101 cs=0 l=1 rev1=1 crypto rx=0x7f72580057c0 tx=0x7f725802e040 comp rx=0 tx=0).stop 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7268105380 msgr2=0x7f726810dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7268105380 0x7f726810dd70 secure :-1 s=READY pgs=3099 cs=0 l=1 rev1=1 crypto rx=0x7f725c00d6a0 tx=0x7f725c00db60 comp rx=0 tx=0).stop 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 shutdown_connections 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f725006da30 0x7f725006fef0 unknown :-1 s=CLOSED pgs=4101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f726810e2b0 0x7f72681a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7268105380 0x7f726810dd70 unknown :-1 s=CLOSED pgs=3099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 --2- 172.21.15.100:0/1578550965 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f72681002b0 0x7f726810d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 >> 172.21.15.100:0/1578550965 conn(0x7f72680771a0 msgr2=0x7f72680fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 shutdown_connections 2024-02-14T18:19:39.833 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.831+0000 7f726cd77640 1 -- 172.21.15.100:0/1578550965 wait complete. 2024-02-14T18:19:39.841 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool application enable 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d rados 2024-02-14T18:19:39.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.923+0000 7fac06a9d640 1 -- 172.21.15.100:0/580419757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac00105090 msgr2=0x7fac00105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:39.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.923+0000 7fac06a9d640 1 --2- 172.21.15.100:0/580419757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac00105090 0x7fac00105470 secure :-1 s=READY pgs=3100 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009900 tx=0x7fabf002bc50 comp rx=0 tx=0).stop 2024-02-14T18:19:39.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.923+0000 7fac06a9d640 1 -- 172.21.15.100:0/580419757 shutdown_connections 2024-02-14T18:19:39.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.923+0000 7fac06a9d640 1 --2- 172.21.15.100:0/580419757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac001015c0 0x7fac0010cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:39.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.923+0000 7fac06a9d640 1 --2- 172.21.15.100:0/580419757 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac00100c00 0x7fac00101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:39.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.923+0000 7fac06a9d640 1 --2- 172.21.15.100:0/580419757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac00105090 0x7fac00105470 unknown :-1 s=CLOSED pgs=3100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:39.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.923+0000 7fac06a9d640 1 -- 172.21.15.100:0/580419757 >> 172.21.15.100:0/580419757 conn(0x7fac00075360 msgr2=0x7fac00075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:39.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.923+0000 7fac06a9d640 1 -- 172.21.15.100:0/580419757 shutdown_connections 2024-02-14T18:19:39.927 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.923+0000 7fac06a9d640 1 -- 172.21.15.100:0/580419757 wait complete. 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac06a9d640 1 Processor -- start 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac06a9d640 1 -- start start 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac06a9d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac00100c00 0x7fac0019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac06a9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac001015c0 0x7fac0019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac06a9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac00105090 0x7fac00196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac06a9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00197090 con 0x7fac00105090 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac06a9d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00197200 con 0x7fac001015c0 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac06a9d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac001973c0 con 0x7fac00100c00 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac04839640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac00100c00 0x7fac0019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac0503a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac00105090 0x7fac00196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac04839640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac00100c00 0x7fac0019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:44030/0 (socket says 172.21.15.100:44030) 2024-02-14T18:19:39.928 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac04839640 1 -- 172.21.15.100:0/1604088479 learned_addr learned my addr 172.21.15.100:0/1604088479 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac04839640 1 -- 172.21.15.100:0/1604088479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac001015c0 msgr2=0x7fac0019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fabfffff640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac001015c0 0x7fac0019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac04839640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac001015c0 0x7fac0019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac04839640 1 -- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac00105090 msgr2=0x7fac00196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac04839640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac00105090 0x7fac00196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac04839640 1 -- 172.21.15.100:0/1604088479 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4009490 con 0x7fac00100c00 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac0503a640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac00105090 0x7fac00196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.924+0000 7fac04839640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac00100c00 0x7fac0019c890 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 crypto rx=0x7fabf0005940 tx=0x7fabf00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.925+0000 7fabfdffb640 1 -- 172.21.15.100:0/1604088479 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf00044e0 con 0x7fac00100c00 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.925+0000 7fabfdffb640 1 -- 172.21.15.100:0/1604088479 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fabf0003c80 con 0x7fac00100c00 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.925+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf0009560 con 0x7fac00100c00 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.925+0000 7fabfdffb640 1 -- 172.21.15.100:0/1604088479 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0005540 con 0x7fac00100c00 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.925+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac001a3750 con 0x7fac00100c00 2024-02-14T18:19:39.929 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.926+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac000684b0 con 0x7fac00100c00 2024-02-14T18:19:39.930 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.926+0000 7fabfdffb640 1 -- 172.21.15.100:0/1604088479 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7fabf0004d90 con 0x7fac00100c00 2024-02-14T18:19:39.930 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.927+0000 7fabfdffb640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fabe806da30 0x7fabe806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:39.930 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.927+0000 7fabfffff640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fabe806da30 0x7fabe806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:39.930 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.927+0000 7fabfdffb640 1 -- 172.21.15.100:0/1604088479 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(784..784 src has 254..784) v4 ==== 9422+0+0 (secure 0 0 0) 0x7fabf00b0490 con 0x7fac00100c00 2024-02-14T18:19:39.930 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.927+0000 7fabfffff640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fabe806da30 0x7fabe806fef0 secure :-1 s=READY pgs=4102 cs=0 l=1 rev1=1 crypto rx=0x7fac000ff5a0 tx=0x7fabf4005f90 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:39.932 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:39.932+0000 7fabfdffb640 1 -- 172.21.15.100:0/1604088479 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fabf00790e0 con 0x7fac00100c00 2024-02-14T18:19:40.107 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:40.106+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"} v 0) v1 -- 0x7fac001085c0 con 0x7fac00100c00 2024-02-14T18:19:40.550 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:19:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:19:40.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1578550965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]: dispatch 2024-02-14T18:19:40.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1578550965' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]': finished 2024-02-14T18:19:40.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[30138]: osdmap e784: 8 total, 8 up, 8 in 2024-02-14T18:19:40.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1578550965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]: dispatch 2024-02-14T18:19:40.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1604088479' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:40.551 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:40.551 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1578550965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]: dispatch 2024-02-14T18:19:40.552 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1578550965' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]': finished 2024-02-14T18:19:40.552 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[39557]: osdmap e784: 8 total, 8 up, 8 in 2024-02-14T18:19:40.552 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1578550965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]: dispatch 2024-02-14T18:19:40.552 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1604088479' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:40.552 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:40 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1578550965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]: dispatch 2024-02-14T18:19:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1578550965' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]': finished 2024-02-14T18:19:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:40 smithi179 ceph-mon[33633]: osdmap e784: 8 total, 8 up, 8 in 2024-02-14T18:19:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1578550965' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pg_num": 12}]: dispatch 2024-02-14T18:19:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:40 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1604088479' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:40.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:40 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:40.731 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:40.730+0000 7fabfdffb640 1 -- 172.21.15.100:0/1604088479 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]=0 enabled application 'rados' on pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' v785) v1 ==== 213+0+0 (secure 0 0 0) 0x7fabf007c7a0 con 0x7fac00100c00 2024-02-14T18:19:40.845 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:40.844+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"} v 0) v1 -- 0x7fac00110980 con 0x7fac00100c00 2024-02-14T18:19:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:41 smithi179 ceph-mon[33633]: pgmap v1736: 188 pgs: 12 unknown, 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:41 smithi179 ceph-mon[33633]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:19:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:41 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]': finished 2024-02-14T18:19:41.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:41 smithi179 ceph-mon[33633]: osdmap e785: 8 total, 8 up, 8 in 2024-02-14T18:19:41.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:41 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1604088479' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:41.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:41 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[30138]: pgmap v1736: 188 pgs: 12 unknown, 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[30138]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:19:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]': finished 2024-02-14T18:19:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[30138]: osdmap e785: 8 total, 8 up, 8 in 2024-02-14T18:19:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1604088479' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:41.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[39557]: pgmap v1736: 188 pgs: 12 unknown, 176 active+clean; 720 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[39557]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:19:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]': finished 2024-02-14T18:19:41.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[39557]: osdmap e785: 8 total, 8 up, 8 in 2024-02-14T18:19:41.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1604088479' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:41.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:41 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]: dispatch 2024-02-14T18:19:41.732 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.731+0000 7fabfdffb640 1 -- 172.21.15.100:0/1604088479 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]=0 enabled application 'rados' on pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' v786) v1 ==== 213+0+0 (secure 0 0 0) 0x7fabf00430d0 con 0x7fac00100c00 2024-02-14T18:19:41.732 INFO:tasks.workunit.client.0.smithi100.stderr:enabled application 'rados' on pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fabe806da30 msgr2=0x7fabe806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fabe806da30 0x7fabe806fef0 secure :-1 s=READY pgs=4102 cs=0 l=1 rev1=1 crypto rx=0x7fac000ff5a0 tx=0x7fabf4005f90 comp rx=0 tx=0).stop 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac00100c00 msgr2=0x7fac0019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac00100c00 0x7fac0019c890 secure :-1 s=READY pgs=3141 cs=0 l=1 rev1=1 crypto rx=0x7fabf0005940 tx=0x7fabf00042f0 comp rx=0 tx=0).stop 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 shutdown_connections 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fabe806da30 0x7fabe806fef0 unknown :-1 s=CLOSED pgs=4102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac00105090 0x7fac00196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac001015c0 0x7fac0019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 --2- 172.21.15.100:0/1604088479 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fac00100c00 0x7fac0019c890 unknown :-1 s=CLOSED pgs=3141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 >> 172.21.15.100:0/1604088479 conn(0x7fac00075360 msgr2=0x7fac000fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:41.736 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 shutdown_connections 2024-02-14T18:19:41.737 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.735+0000 7fac06a9d640 1 -- 172.21.15.100:0/1604088479 wait complete. 2024-02-14T18:19:41.747 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d max_objects 10 2024-02-14T18:19:41.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.833+0000 7ff04a29d640 1 -- 172.21.15.100:0/130263822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0441002b0 msgr2=0x7ff044100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:41.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.833+0000 7ff04a29d640 1 --2- 172.21.15.100:0/130263822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0441002b0 0x7ff044100690 secure :-1 s=READY pgs=3142 cs=0 l=1 rev1=1 crypto rx=0x7ff034009900 tx=0x7ff03402bc30 comp rx=0 tx=0).stop 2024-02-14T18:19:41.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.834+0000 7ff04a29d640 1 -- 172.21.15.100:0/130263822 shutdown_connections 2024-02-14T18:19:41.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.834+0000 7ff04a29d640 1 --2- 172.21.15.100:0/130263822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff044105380 0x7ff04410cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:41.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.834+0000 7ff04a29d640 1 --2- 172.21.15.100:0/130263822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff044100c60 0x7ff044104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:41.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.834+0000 7ff04a29d640 1 --2- 172.21.15.100:0/130263822 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff0441002b0 0x7ff044100690 unknown :-1 s=CLOSED pgs=3142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:41.835 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.834+0000 7ff04a29d640 1 -- 172.21.15.100:0/130263822 >> 172.21.15.100:0/130263822 conn(0x7ff0440771a0 msgr2=0x7ff0440775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:41.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.834+0000 7ff04a29d640 1 -- 172.21.15.100:0/130263822 shutdown_connections 2024-02-14T18:19:41.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.834+0000 7ff04a29d640 1 -- 172.21.15.100:0/130263822 wait complete. 2024-02-14T18:19:41.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.834+0000 7ff04a29d640 1 Processor -- start 2024-02-14T18:19:41.836 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.834+0000 7ff04a29d640 1 -- start start 2024-02-14T18:19:41.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04a29d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0441002b0 0x7ff044196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:41.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04a29d640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff044105380 0x7ff0441972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:41.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04a29d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff04419b660 0x7ff04419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:41.838 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04a29d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04419c050 con 0x7ff04419b660 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04a29d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04419c1c0 con 0x7ff0441002b0 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04a29d640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04419c330 con 0x7ff044105380 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff044105380 0x7ff0441972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04883a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff04419b660 0x7ff04419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04883a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff04419b660 0x7ff04419bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:39922/0 (socket says 172.21.15.100:39922) 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0437fe640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff044105380 0x7ff0441972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:44034/0 (socket says 172.21.15.100:44034) 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04883a640 1 -- 172.21.15.100:0/794133597 learned_addr learned my addr 172.21.15.100:0/794133597 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff043fff640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0441002b0 0x7ff044196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0437fe640 1 -- 172.21.15.100:0/794133597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0441002b0 msgr2=0x7ff044196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0437fe640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0441002b0 0x7ff044196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0437fe640 1 -- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff04419b660 msgr2=0x7ff04419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0437fe640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff04419b660 0x7ff04419bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0437fe640 1 -- 172.21.15.100:0/794133597 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c009840 con 0x7ff044105380 2024-02-14T18:19:41.839 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04883a640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff04419b660 0x7ff04419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff043fff640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0441002b0 0x7ff044196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0437fe640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff044105380 0x7ff0441972e0 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7ff04406a8b0 tx=0x7ff03800dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0417fa640 1 -- 172.21.15.100:0/794133597 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038004020 con 0x7ff044105380 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff034009560 con 0x7ff044105380 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0417fa640 1 -- 172.21.15.100:0/794133597 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff0380099d0 con 0x7ff044105380 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff0417fa640 1 -- 172.21.15.100:0/794133597 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380047a0 con 0x7ff044105380 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.835+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0441a3c30 con 0x7ff044105380 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.836+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff044068060 con 0x7ff044105380 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.837+0000 7ff0417fa640 1 -- 172.21.15.100:0/794133597 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7ff038004180 con 0x7ff044105380 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.837+0000 7ff0417fa640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff02c06da30 0x7ff02c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.837+0000 7ff043fff640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff02c06da30 0x7ff02c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.837+0000 7ff0417fa640 1 -- 172.21.15.100:0/794133597 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(786..786 src has 254..786) v4 ==== 9435+0+0 (secure 0 0 0) 0x7ff03808dce0 con 0x7ff044105380 2024-02-14T18:19:41.840 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.838+0000 7ff043fff640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff02c06da30 0x7ff02c06fef0 secure :-1 s=READY pgs=4103 cs=0 l=1 rev1=1 crypto rx=0x7ff034005ec0 tx=0x7ff034000bc0 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:19:41.842 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:41.842+0000 7ff0417fa640 1 -- 172.21.15.100:0/794133597 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff038056920 con 0x7ff044105380 2024-02-14T18:19:42.016 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:42.015+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7ff044100c60 con 0x7ff044105380 2024-02-14T18:19:42.738 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:42.737+0000 7ff0417fa640 1 -- 172.21.15.100:0/794133597 <== mon.2 v2:172.21.15.100:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d v787) v1 ==== 223+0+0 (secure 0 0 0) 0x7ff0380170d0 con 0x7ff044105380 2024-02-14T18:19:42.848 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:42.847+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7ff0441107b0 con 0x7ff044105380 2024-02-14T18:19:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]': finished 2024-02-14T18:19:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:42 smithi179 ceph-mon[33633]: osdmap e786: 8 total, 8 up, 8 in 2024-02-14T18:19:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:42 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/794133597' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:42 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:43.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:42 smithi179 ceph-mon[33633]: pgmap v1739: 188 pgs: 188 active+clean; 743 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-14T18:19:43.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]': finished 2024-02-14T18:19:43.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:42 smithi100 ceph-mon[30138]: osdmap e786: 8 total, 8 up, 8 in 2024-02-14T18:19:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:42 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/794133597' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:42 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:43.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:42 smithi100 ceph-mon[30138]: pgmap v1739: 188 pgs: 188 active+clean; 743 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-14T18:19:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "app": "rados"}]': finished 2024-02-14T18:19:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:42 smithi100 ceph-mon[39557]: osdmap e786: 8 total, 8 up, 8 in 2024-02-14T18:19:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:42 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/794133597' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:42 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:43.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:42 smithi100 ceph-mon[39557]: pgmap v1739: 188 pgs: 188 active+clean; 743 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-14T18:19:43.741 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.740+0000 7ff0417fa640 1 -- 172.21.15.100:0/794133597 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d v788) v1 ==== 223+0+0 (secure 0 0 0) 0x7ff038059fe0 con 0x7ff044105380 2024-02-14T18:19:43.741 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_objects = 10 for pool 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d 2024-02-14T18:19:43.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff02c06da30 msgr2=0x7ff02c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:43.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff02c06da30 0x7ff02c06fef0 secure :-1 s=READY pgs=4103 cs=0 l=1 rev1=1 crypto rx=0x7ff034005ec0 tx=0x7ff034000bc0 comp rx=0 tx=0).stop 2024-02-14T18:19:43.744 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff044105380 msgr2=0x7ff0441972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:19:43.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff044105380 0x7ff0441972e0 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7ff04406a8b0 tx=0x7ff03800dbb0 comp rx=0 tx=0).stop 2024-02-14T18:19:43.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 shutdown_connections 2024-02-14T18:19:43.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7ff02c06da30 0x7ff02c06fef0 unknown :-1 s=CLOSED pgs=4103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:43.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff04419b660 0x7ff04419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:43.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7ff044105380 0x7ff0441972e0 unknown :-1 s=CLOSED pgs=3143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:43.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 --2- 172.21.15.100:0/794133597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0441002b0 0x7ff044196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:19:43.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 >> 172.21.15.100:0/794133597 conn(0x7ff0440771a0 msgr2=0x7ff0440fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:19:43.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 shutdown_connections 2024-02-14T18:19:43.745 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:19:43.743+0000 7ff04a29d640 1 -- 172.21.15.100:0/794133597 wait complete. 2024-02-14T18:19:43.746 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:19:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:19:43.762 INFO:tasks.workunit.client.0.smithi100.stderr:+ sleep 30 2024-02-14T18:19:44.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:19:44.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:43 smithi179 ceph-mon[33633]: osdmap e787: 8 total, 8 up, 8 in 2024-02-14T18:19:44.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:43 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/794133597' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:44.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:43 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:44.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:43 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:44.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:44.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[30138]: osdmap e787: 8 total, 8 up, 8 in 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/794133597' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[39557]: osdmap e787: 8 total, 8 up, 8 in 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/794133597' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]: dispatch 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:19:44.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:45.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:45.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:44 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:19:45.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:44 smithi179 ceph-mon[33633]: osdmap e788: 8 total, 8 up, 8 in 2024-02-14T18:19:45.071 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:44 smithi179 ceph-mon[33633]: pgmap v1742: 188 pgs: 188 active+clean; 743 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-14T18:19:45.149 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:45.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:44 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:19:45.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:44 smithi100 ceph-mon[30138]: osdmap e788: 8 total, 8 up, 8 in 2024-02-14T18:19:45.150 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:44 smithi100 ceph-mon[30138]: pgmap v1742: 188 pgs: 188 active+clean; 743 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-14T18:19:45.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:45.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:44 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "field": "max_objects", "val": "10"}]': finished 2024-02-14T18:19:45.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:44 smithi100 ceph-mon[39557]: osdmap e788: 8 total, 8 up, 8 in 2024-02-14T18:19:45.150 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:44 smithi100 ceph-mon[39557]: pgmap v1742: 188 pgs: 188 active+clean; 743 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-14T18:19:46.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:46 smithi100 ceph-mon[30138]: pgmap v1743: 188 pgs: 188 active+clean; 743 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 925 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-02-14T18:19:46.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:46 smithi100 ceph-mon[39557]: pgmap v1743: 188 pgs: 188 active+clean; 743 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 925 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-02-14T18:19:46.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:46 smithi179 ceph-mon[33633]: pgmap v1743: 188 pgs: 188 active+clean; 743 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 925 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-02-14T18:19:48.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:48 smithi100 ceph-mon[30138]: pgmap v1744: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 783 B/s rd, 0 op/s 2024-02-14T18:19:48.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:48 smithi100 ceph-mon[39557]: pgmap v1744: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 783 B/s rd, 0 op/s 2024-02-14T18:19:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:48 smithi179 ceph-mon[33633]: pgmap v1744: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 783 B/s rd, 0 op/s 2024-02-14T18:19:49.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:49 smithi100 ceph-mon[30138]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:19:49.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:49 smithi100 ceph-mon[39557]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:19:49.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:49 smithi179 ceph-mon[33633]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-14T18:19:50.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:50 smithi100 ceph-mon[30138]: pgmap v1745: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T18:19:50.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:19:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:19:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:19:50.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:50 smithi100 ceph-mon[39557]: pgmap v1745: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T18:19:50.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:50 smithi179 ceph-mon[33633]: pgmap v1745: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-14T18:19:52.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:52 smithi100 ceph-mon[30138]: pgmap v1746: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-14T18:19:52.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:52 smithi100 ceph-mon[39557]: pgmap v1746: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-14T18:19:52.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:52 smithi179 ceph-mon[33633]: pgmap v1746: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-14T18:19:53.722 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:19:53 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:19:54.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:54.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:54 smithi100 ceph-mon[30138]: pgmap v1747: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 972 B/s rd, 0 op/s 2024-02-14T18:19:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:54.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:54 smithi100 ceph-mon[39557]: pgmap v1747: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 972 B/s rd, 0 op/s 2024-02-14T18:19:54.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:19:54.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:54 smithi179 ceph-mon[33633]: pgmap v1747: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 972 B/s rd, 0 op/s 2024-02-14T18:19:56.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:56 smithi100 ceph-mon[30138]: pgmap v1748: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:56.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:56 smithi100 ceph-mon[39557]: pgmap v1748: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:56.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:56 smithi179 ceph-mon[33633]: pgmap v1748: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:19:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:58.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:19:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:19:59 smithi179 ceph-mon[33633]: pgmap v1749: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:59.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:19:59 smithi100 ceph-mon[30138]: pgmap v1749: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:19:59.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:19:59 smithi100 ceph-mon[39557]: pgmap v1749: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:00 smithi100 ceph-mon[30138]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-14T18:20:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:00 smithi100 ceph-mon[30138]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-14T18:20:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:00 smithi100 ceph-mon[30138]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:20:00.494 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:00 smithi100 ceph-mon[30138]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:20:00.494 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:00 smithi100 ceph-mon[39557]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-14T18:20:00.494 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:00 smithi100 ceph-mon[39557]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-14T18:20:00.494 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:00 smithi100 ceph-mon[39557]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:20:00.494 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:00 smithi100 ceph-mon[39557]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:20:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:00 smithi179 ceph-mon[33633]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-14T18:20:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:00 smithi179 ceph-mon[33633]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-14T18:20:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:00 smithi179 ceph-mon[33633]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-14T18:20:00.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:00 smithi179 ceph-mon[33633]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-14T18:20:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:20:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:20:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:20:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:01 smithi179 ceph-mon[33633]: pgmap v1750: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:01 smithi100 ceph-mon[39557]: pgmap v1750: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:01 smithi100 ceph-mon[30138]: pgmap v1750: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:02.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:02 smithi100 ceph-mon[30138]: pgmap v1751: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:02.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:02 smithi100 ceph-mon[39557]: pgmap v1751: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:02.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:02 smithi179 ceph-mon[33633]: pgmap v1751: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:03.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:03 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:20:03.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:03 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:20:03.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:03 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:20:03.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:20:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:20:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:04 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:04 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:20:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:04 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:20:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:04 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:20:04.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:04 smithi100 ceph-mon[30138]: pgmap v1752: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:04 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:04 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:20:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:04 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:20:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:04 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:20:04.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:04 smithi100 ceph-mon[39557]: pgmap v1752: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:04 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:04 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:20:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:04 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:20:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:04 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:20:04.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:04 smithi179 ceph-mon[33633]: pgmap v1752: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:06.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:06 smithi100 ceph-mon[30138]: pgmap v1753: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:06.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:06 smithi100 ceph-mon[39557]: pgmap v1753: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:06.824 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:06 smithi179 ceph-mon[33633]: pgmap v1753: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:08.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:08 smithi100 ceph-mon[30138]: pgmap v1754: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:08.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:08 smithi100 ceph-mon[39557]: pgmap v1754: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:08.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:08 smithi179 ceph-mon[33633]: pgmap v1754: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:10.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:10 smithi100 ceph-mon[30138]: pgmap v1755: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:10.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:10 smithi100 ceph-mon[39557]: pgmap v1755: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:10.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:20:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:20:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:20:10.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:10 smithi179 ceph-mon[33633]: pgmap v1755: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:12.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:12 smithi100 ceph-mon[30138]: pgmap v1756: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:12.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:12 smithi100 ceph-mon[39557]: pgmap v1756: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:12.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:12 smithi179 ceph-mon[33633]: pgmap v1756: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:13.764 INFO:tasks.workunit.client.0.smithi100.stderr:++ seq 1 10 2024-02-14T18:20:13.765 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:20:13.765 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d put obj1 /etc/passwd 2024-02-14T18:20:13.806 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:20:13.807 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d put obj2 /etc/passwd 2024-02-14T18:20:13.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:13.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:20:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:20:13.844 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:20:13.844 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d put obj3 /etc/passwd 2024-02-14T18:20:13.880 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:20:13.880 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d put obj4 /etc/passwd 2024-02-14T18:20:13.915 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:20:13.916 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d put obj5 /etc/passwd 2024-02-14T18:20:13.950 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:20:13.950 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d put obj6 /etc/passwd 2024-02-14T18:20:13.984 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:20:13.985 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d put obj7 /etc/passwd 2024-02-14T18:20:14.018 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:20:14.018 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d put obj8 /etc/passwd 2024-02-14T18:20:14.053 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:20:14.053 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d put obj9 /etc/passwd 2024-02-14T18:20:14.087 INFO:tasks.workunit.client.0.smithi100.stderr:+ for f in `seq 1 10` 2024-02-14T18:20:14.087 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d put obj10 /etc/passwd 2024-02-14T18:20:14.122 INFO:tasks.workunit.client.0.smithi100.stderr:+ sleep 30 2024-02-14T18:20:14.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:14 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:14.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:14 smithi100 ceph-mon[39557]: pgmap v1757: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:14 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:14.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:14 smithi100 ceph-mon[30138]: pgmap v1757: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:14 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:14.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:14 smithi179 ceph-mon[33633]: pgmap v1757: 188 pgs: 188 active+clean; 743 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:16.651 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:16 smithi100 ceph-mon[30138]: pgmap v1758: 188 pgs: 188 active+clean; 789 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:20:16.652 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:16 smithi100 ceph-mon[39557]: pgmap v1758: 188 pgs: 188 active+clean; 789 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:20:16.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:16 smithi179 ceph-mon[33633]: pgmap v1758: 188 pgs: 188 active+clean; 789 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-02-14T18:20:18.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:18 smithi100 ceph-mon[30138]: pgmap v1759: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-14T18:20:18.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:18 smithi100 ceph-mon[39557]: pgmap v1759: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-14T18:20:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:18 smithi179 ceph-mon[33633]: pgmap v1759: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-14T18:20:19.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:19 smithi100 ceph-mon[30138]: pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' is full (reached quota's max_objects: 10) 2024-02-14T18:20:19.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:19 smithi100 ceph-mon[30138]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:20:19.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:19 smithi100 ceph-mon[30138]: osdmap e789: 8 total, 8 up, 8 in 2024-02-14T18:20:19.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:19 smithi100 ceph-mon[39557]: pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' is full (reached quota's max_objects: 10) 2024-02-14T18:20:19.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:19 smithi100 ceph-mon[39557]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:20:19.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:19 smithi100 ceph-mon[39557]: osdmap e789: 8 total, 8 up, 8 in 2024-02-14T18:20:19.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:19 smithi179 ceph-mon[33633]: pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' is full (reached quota's max_objects: 10) 2024-02-14T18:20:19.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:19 smithi179 ceph-mon[33633]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-14T18:20:19.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:19 smithi179 ceph-mon[33633]: osdmap e789: 8 total, 8 up, 8 in 2024-02-14T18:20:20.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:20 smithi100 ceph-mon[30138]: pgmap v1761: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:20:20.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:20 smithi100 ceph-mon[39557]: pgmap v1761: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:20:20.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:20:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:20:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:20:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:20 smithi179 ceph-mon[33633]: pgmap v1761: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:20:22.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:22 smithi100 ceph-mon[30138]: pgmap v1762: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:20:22.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:22 smithi100 ceph-mon[39557]: pgmap v1762: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:20:22.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:22 smithi179 ceph-mon[33633]: pgmap v1762: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:20:23.722 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:20:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:20:24.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:24 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:24.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:24 smithi100 ceph-mon[30138]: pgmap v1763: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:20:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:24 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:24.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:24 smithi100 ceph-mon[39557]: pgmap v1763: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:20:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:24 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:24 smithi179 ceph-mon[33633]: pgmap v1763: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-14T18:20:26.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:26 smithi100 ceph-mon[39557]: pgmap v1764: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-02-14T18:20:26.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:26 smithi100 ceph-mon[30138]: pgmap v1764: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-02-14T18:20:26.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:26 smithi179 ceph-mon[33633]: pgmap v1764: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-02-14T18:20:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:29 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:20:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:29 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:29.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:29 smithi179 ceph-mon[33633]: pgmap v1765: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:20:29.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:29 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:20:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:29 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:29.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:29 smithi100 ceph-mon[39557]: pgmap v1765: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:20:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:29 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:20:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:29 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:29.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:29 smithi100 ceph-mon[30138]: pgmap v1765: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:20:30.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:30 smithi100 ceph-mon[30138]: pgmap v1766: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 886 B/s rd, 0 op/s 2024-02-14T18:20:30.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:30 smithi100 ceph-mon[39557]: pgmap v1766: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 886 B/s rd, 0 op/s 2024-02-14T18:20:30.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:20:30 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:20:30] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:20:30.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:30 smithi179 ceph-mon[33633]: pgmap v1766: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 886 B/s rd, 0 op/s 2024-02-14T18:20:32.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:32 smithi100 ceph-mon[39557]: pgmap v1767: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:32.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:32 smithi100 ceph-mon[30138]: pgmap v1767: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:32.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:32 smithi179 ceph-mon[33633]: pgmap v1767: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:33.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:20:33 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:20:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:34 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:34.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:34 smithi100 ceph-mon[39557]: pgmap v1768: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:20:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:34 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:34.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:34 smithi100 ceph-mon[30138]: pgmap v1768: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:20:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:34 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:34.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:34 smithi179 ceph-mon[33633]: pgmap v1768: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:20:36.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:36 smithi100 ceph-mon[30138]: pgmap v1769: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:36.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:36 smithi100 ceph-mon[39557]: pgmap v1769: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:36.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:36 smithi179 ceph-mon[33633]: pgmap v1769: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:38.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:38 smithi100 ceph-mon[30138]: pgmap v1770: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:38.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:38 smithi100 ceph-mon[39557]: pgmap v1770: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:38.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:38 smithi179 ceph-mon[33633]: pgmap v1770: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:40.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:40 smithi100 ceph-mon[39557]: pgmap v1771: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:40.650 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:20:40 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:20:40] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:20:40.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:40 smithi100 ceph-mon[30138]: pgmap v1771: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:40.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:40 smithi179 ceph-mon[33633]: pgmap v1771: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:42.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:42 smithi100 ceph-mon[39557]: pgmap v1772: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:42.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:42 smithi100 ceph-mon[30138]: pgmap v1772: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:42.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:42 smithi179 ceph-mon[33633]: pgmap v1772: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:20:43.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:43 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:43.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:43 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:43.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:43 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:43.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:20:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:20:44.123 INFO:tasks.workunit.client.0.smithi100.stderr:+ rados -p 8892ea91-15b7-4906-9324-dc08ab1d48a5 put threemore /etc/passwd 2024-02-14T18:20:44.164 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 8892ea91-15b7-4906-9324-dc08ab1d48a5 max_bytes 0 2024-02-14T18:20:44.254 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.251+0000 7fb299e40640 1 -- 172.21.15.100:0/2142023331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb294106480 msgr2=0x7fb29410e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.251+0000 7fb299e40640 1 --2- 172.21.15.100:0/2142023331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb294106480 0x7fb29410e050 secure :-1 s=READY pgs=3156 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009be0 tx=0x7fb28c02bef0 comp rx=0 tx=0).stop 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.251+0000 7fb299e40640 1 -- 172.21.15.100:0/2142023331 shutdown_connections 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.251+0000 7fb299e40640 1 --2- 172.21.15.100:0/2142023331 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb294106480 0x7fb29410e050 unknown :-1 s=CLOSED pgs=3156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.251+0000 7fb299e40640 1 --2- 172.21.15.100:0/2142023331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb294101d60 0x7fb294105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.251+0000 7fb299e40640 1 --2- 172.21.15.100:0/2142023331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb2941013b0 0x7fb294101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.251+0000 7fb299e40640 1 -- 172.21.15.100:0/2142023331 >> 172.21.15.100:0/2142023331 conn(0x7fb2940fcc10 msgr2=0x7fb2940ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.251+0000 7fb299e40640 1 -- 172.21.15.100:0/2142023331 shutdown_connections 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.251+0000 7fb299e40640 1 -- 172.21.15.100:0/2142023331 wait complete. 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb299e40640 1 Processor -- start 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb299e40640 1 -- start start 2024-02-14T18:20:44.255 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb299e40640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2941013b0 0x7fb294196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb299e40640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb294106480 0x7fb294197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb299e40640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb29419b600 0x7fb29419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb299e40640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29419bff0 con 0x7fb294106480 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb299e40640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29419c160 con 0x7fb2941013b0 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb299e40640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29419c2d0 con 0x7fb29419b600 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb292ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb294106480 0x7fb294197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb293fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb29419b600 0x7fb29419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb293fff640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb29419b600 0x7fb29419bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:52130/0 (socket says 172.21.15.100:52130) 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb293fff640 1 -- 172.21.15.100:0/3143375028 learned_addr learned my addr 172.21.15.100:0/3143375028 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb2937fe640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2941013b0 0x7fb294196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb293fff640 1 -- 172.21.15.100:0/3143375028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2941013b0 msgr2=0x7fb294196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb293fff640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2941013b0 0x7fb294196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:44.256 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb293fff640 1 -- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb294106480 msgr2=0x7fb294197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:20:44.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb293fff640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb294106480 0x7fb294197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:44.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb293fff640 1 -- 172.21.15.100:0/3143375028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb284009560 con 0x7fb29419b600 2024-02-14T18:20:44.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.252+0000 7fb292ffd640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb294106480 0x7fb294197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:20:44.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.253+0000 7fb293fff640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb29419b600 0x7fb29419bab0 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 crypto rx=0x7fb28c003ce0 tx=0x7fb28c003d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:20:44.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.253+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 <== mon.2 v2:172.21.15.100:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c033b10 con 0x7fb29419b600 2024-02-14T18:20:44.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.253+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28c009840 con 0x7fb29419b600 2024-02-14T18:20:44.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.253+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 <== mon.2 v2:172.21.15.100:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb28c033c70 con 0x7fb29419b600 2024-02-14T18:20:44.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.253+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 <== mon.2 v2:172.21.15.100:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c034710 con 0x7fb29419b600 2024-02-14T18:20:44.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.253+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2941a3bd0 con 0x7fb29419b600 2024-02-14T18:20:44.257 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.254+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2940684b0 con 0x7fb29419b600 2024-02-14T18:20:44.258 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.254+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 <== mon.2 v2:172.21.15.100:3301/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7fb28c004040 con 0x7fb29419b600 2024-02-14T18:20:44.258 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.254+0000 7fb290ff9640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb27c06da30 0x7fb27c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:20:44.259 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.255+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 <== mon.2 v2:172.21.15.100:3301/0 5 ==== osd_map(789..789 src has 254..789) v4 ==== 9435+0+0 (secure 0 0 0) 0x7fb28c0b1030 con 0x7fb29419b600 2024-02-14T18:20:44.259 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.255+0000 7fb2937fe640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb27c06da30 0x7fb27c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:20:44.259 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.255+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=790}) v3 -- 0x7fb27c079cc0 con 0x7fb29419b600 2024-02-14T18:20:44.259 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.255+0000 7fb2937fe640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb27c06da30 0x7fb27c06fef0 secure :-1 s=READY pgs=4115 cs=0 l=1 rev1=1 crypto rx=0x7fb284005fd0 tx=0x7fb284009260 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:20:44.260 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.259+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 <== mon.2 v2:172.21.15.100:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb28c079c70 con 0x7fb29419b600 2024-02-14T18:20:44.436 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:44.435+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fb294101d60 con 0x7fb29419b600 2024-02-14T18:20:44.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:44 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:44.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:44 smithi100 ceph-mon[39557]: pgmap v1773: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:20:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:44 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:44.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:44 smithi100 ceph-mon[30138]: pgmap v1773: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:20:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:44 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:44.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:44 smithi179 ceph-mon[33633]: pgmap v1773: 188 pgs: 188 active+clean; 858 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-14T18:20:45.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:45.388+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 <== mon.2 v2:172.21.15.100:3301/0 7 ==== osd_map(790..790 src has 254..790) v4 ==== 628+0+0 (secure 0 0 0) 0x7fb28c07e0d0 con 0x7fb29419b600 2024-02-14T18:20:45.389 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:45.388+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=791}) v3 -- 0x7fb27c07a830 con 0x7fb29419b600 2024-02-14T18:20:45.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:45.391+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 <== mon.2 v2:172.21.15.100:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v790) v1 ==== 217+0+0 (secure 0 0 0) 0x7fb28c0430d0 con 0x7fb29419b600 2024-02-14T18:20:45.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:45.501+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fb294101f70 con 0x7fb29419b600 2024-02-14T18:20:45.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:45 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3143375028' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:45.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:45 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:45 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3143375028' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:45.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:45 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:45.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:45 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3143375028' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:45.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:45 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:46.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.391+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 <== mon.2 v2:172.21.15.100:3301/0 9 ==== osd_map(791..791 src has 254..791) v4 ==== 628+0+0 (secure 0 0 0) 0x7fb28c081b20 con 0x7fb29419b600 2024-02-14T18:20:46.392 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.391+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_subscribe({osdmap=792}) v3 -- 0x7fb27c07aca0 con 0x7fb29419b600 2024-02-14T18:20:46.397 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.397+0000 7fb290ff9640 1 -- 172.21.15.100:0/3143375028 <== mon.2 v2:172.21.15.100:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v791) v1 ==== 217+0+0 (secure 0 0 0) 0x7fb28c07d330 con 0x7fb29419b600 2024-02-14T18:20:46.398 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_bytes = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 2024-02-14T18:20:46.401 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.400+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb27c06da30 msgr2=0x7fb27c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.400+0000 7fb299e40640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb27c06da30 0x7fb27c06fef0 secure :-1 s=READY pgs=4115 cs=0 l=1 rev1=1 crypto rx=0x7fb284005fd0 tx=0x7fb284009260 comp rx=0 tx=0).stop 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.400+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb29419b600 msgr2=0x7fb29419bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.400+0000 7fb299e40640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb29419b600 0x7fb29419bab0 secure :-1 s=READY pgs=3157 cs=0 l=1 rev1=1 crypto rx=0x7fb28c003ce0 tx=0x7fb28c003d10 comp rx=0 tx=0).stop 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.400+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 shutdown_connections 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.400+0000 7fb299e40640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb27c06da30 0x7fb27c06fef0 unknown :-1 s=CLOSED pgs=4115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.400+0000 7fb299e40640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb29419b600 0x7fb29419bab0 unknown :-1 s=CLOSED pgs=3157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.400+0000 7fb299e40640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb294106480 0x7fb294197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.400+0000 7fb299e40640 1 --2- 172.21.15.100:0/3143375028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2941013b0 0x7fb294196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.400+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 >> 172.21.15.100:0/3143375028 conn(0x7fb2940fcc10 msgr2=0x7fb2940fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.401+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 shutdown_connections 2024-02-14T18:20:46.402 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.401+0000 7fb299e40640 1 -- 172.21.15.100:0/3143375028 wait complete. 2024-02-14T18:20:46.411 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool set-quota 8892ea91-15b7-4906-9324-dc08ab1d48a5 max_objects 0 2024-02-14T18:20:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 -- 172.21.15.100:0/2278153512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a01002b0 msgr2=0x7f63a0100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:20:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 --2- 172.21.15.100:0/2278153512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a01002b0 0x7f63a0100690 secure :-1 s=READY pgs=3122 cs=0 l=1 rev1=1 crypto rx=0x7f639c009890 tx=0x7f639c02bb70 comp rx=0 tx=0).stop 2024-02-14T18:20:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 -- 172.21.15.100:0/2278153512 shutdown_connections 2024-02-14T18:20:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 --2- 172.21.15.100:0/2278153512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63a0105380 0x7f63a010cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 --2- 172.21.15.100:0/2278153512 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63a0100c60 0x7f63a0104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 --2- 172.21.15.100:0/2278153512 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a01002b0 0x7f63a0100690 unknown :-1 s=CLOSED pgs=3122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:46.499 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 -- 172.21.15.100:0/2278153512 >> 172.21.15.100:0/2278153512 conn(0x7f63a00771a0 msgr2=0x7f63a00775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 -- 172.21.15.100:0/2278153512 shutdown_connections 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 -- 172.21.15.100:0/2278153512 wait complete. 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 Processor -- start 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 -- start start 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a01002b0 0x7f63a0197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63a0105380 0x7f63a0198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63a01986d0 0x7f63a01a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01a3320 con 0x7f63a01002b0 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01a3490 con 0x7f63a0105380 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a7330640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01a3650 con 0x7f63a01986d0 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.495+0000 7f63a50cc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a01002b0 0x7f63a0197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:20:46.500 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a50cc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a01002b0 0x7f63a0197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52354/0 (socket says 172.21.15.100:52354) 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a50cc640 1 -- 172.21.15.100:0/2432772453 learned_addr learned my addr 172.21.15.100:0/2432772453 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a58cd640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63a01986d0 0x7f63a01a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a50cc640 1 -- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63a01986d0 msgr2=0x7f63a01a2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a50cc640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63a01986d0 0x7f63a01a2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a50cc640 1 -- 172.21.15.100:0/2432772453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63a0105380 msgr2=0x7f63a0198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a50cc640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63a0105380 0x7f63a0198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a50cc640 1 -- 172.21.15.100:0/2432772453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390009560 con 0x7f63a01002b0 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a50cc640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a01002b0 0x7f63a0197c00 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c02c510 con 0x7f63a01002b0 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639c009490 con 0x7f63a01002b0 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f639c02caf0 con 0x7f63a01002b0 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c03d3f0 con 0x7f63a01002b0 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.496+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a01a3c40 con 0x7f63a01002b0 2024-02-14T18:20:46.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.497+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a0068060 con 0x7f63a01002b0 2024-02-14T18:20:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.497+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f639c03b040 con 0x7f63a01002b0 2024-02-14T18:20:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.498+0000 7f63967fc640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f638c06da30 0x7f638c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:20:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.498+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(791..791 src has 254..791) v4 ==== 9435+0+0 (secure 0 0 0) 0x7f639c0afb70 con 0x7f63a01002b0 2024-02-14T18:20:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.498+0000 7f63a48cb640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f638c06da30 0x7f638c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:20:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.498+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=792}) v3 -- 0x7f638c079cc0 con 0x7f63a01002b0 2024-02-14T18:20:46.502 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.498+0000 7f63a48cb640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f638c06da30 0x7f638c06fef0 secure :-1 s=READY pgs=4116 cs=0 l=1 rev1=1 crypto rx=0x7f6390005ab0 tx=0x7f6390005a20 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:20:46.503 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.503+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f639c078860 con 0x7f63a01002b0 2024-02-14T18:20:46.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:20:46.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:46 smithi100 ceph-mon[39557]: osdmap e790: 8 total, 8 up, 8 in 2024-02-14T18:20:46.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:46 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/3143375028' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:46.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:46 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:46.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:46 smithi100 ceph-mon[39557]: pgmap v1775: 188 pgs: 188 active+clean; 870 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-14T18:20:46.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:20:46.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:46 smithi100 ceph-mon[30138]: osdmap e790: 8 total, 8 up, 8 in 2024-02-14T18:20:46.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:46 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/3143375028' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:46.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:46 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:46.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:46 smithi100 ceph-mon[30138]: pgmap v1775: 188 pgs: 188 active+clean; 870 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-14T18:20:46.680 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:46.679+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f63a01a3810 con 0x7f63a01002b0 2024-02-14T18:20:46.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:20:46.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:46 smithi179 ceph-mon[33633]: osdmap e790: 8 total, 8 up, 8 in 2024-02-14T18:20:46.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:46 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/3143375028' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:46.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:46 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-14T18:20:46.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:46 smithi179 ceph-mon[33633]: pgmap v1775: 188 pgs: 188 active+clean; 870 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-14T18:20:47.399 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:47.398+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v792) v1 ==== 221+0+0 (secure 0 0 0) 0x7f639c039020 con 0x7f63a01002b0 2024-02-14T18:20:47.412 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:47.411+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 <== mon.0 v2:172.21.15.100:3300/0 8 ==== osd_map(792..792 src has 254..792) v4 ==== 628+0+0 (secure 0 0 0) 0x7f639c0035f0 con 0x7f63a01002b0 2024-02-14T18:20:47.412 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:47.411+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=793}) v3 -- 0x7f638c07ac20 con 0x7f63a01002b0 2024-02-14T18:20:47.511 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:47.510+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f63a0100c60 con 0x7f63a01002b0 2024-02-14T18:20:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:47 smithi100 ceph-mon[39557]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:20:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:47 smithi100 ceph-mon[39557]: osdmap e791: 8 total, 8 up, 8 in 2024-02-14T18:20:47.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:47 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:20:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:47 smithi100 ceph-mon[30138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:20:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:47 smithi100 ceph-mon[30138]: osdmap e791: 8 total, 8 up, 8 in 2024-02-14T18:20:47.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:47 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:20:47.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:47 smithi179 ceph-mon[33633]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_bytes", "val": "0"}]': finished 2024-02-14T18:20:47.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:47 smithi179 ceph-mon[33633]: osdmap e791: 8 total, 8 up, 8 in 2024-02-14T18:20:47.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:47 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:20:48.406 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.405+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 <== mon.0 v2:172.21.15.100:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 v793) v1 ==== 221+0+0 (secure 0 0 0) 0x7f639c039070 con 0x7f63a01002b0 2024-02-14T18:20:48.406 INFO:tasks.workunit.client.0.smithi100.stderr:set-quota max_objects = 0 for pool 8892ea91-15b7-4906-9324-dc08ab1d48a5 2024-02-14T18:20:48.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63967fc640 1 -- 172.21.15.100:0/2432772453 <== mon.0 v2:172.21.15.100:3300/0 10 ==== osd_map(793..793 src has 254..793) v4 ==== 628+0+0 (secure 0 0 0) 0x7f639c080660 con 0x7f63a01002b0 2024-02-14T18:20:48.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f638c06da30 msgr2=0x7f638c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:20:48.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63a7330640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f638c06da30 0x7f638c06fef0 secure :-1 s=READY pgs=4116 cs=0 l=1 rev1=1 crypto rx=0x7f6390005ab0 tx=0x7f6390005a20 comp rx=0 tx=0).stop 2024-02-14T18:20:48.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a01002b0 msgr2=0x7f63a0197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:20:48.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63a7330640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a01002b0 0x7f63a0197c00 secure :-1 s=READY pgs=3123 cs=0 l=1 rev1=1 crypto rx=0x7f639c000c00 tx=0x7f639c004040 comp rx=0 tx=0).stop 2024-02-14T18:20:48.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 shutdown_connections 2024-02-14T18:20:48.409 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63a7330640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f638c06da30 0x7f638c06fef0 secure :-1 s=CLOSED pgs=4116 cs=0 l=1 rev1=1 crypto rx=0x7f6390005ab0 tx=0x7f6390005a20 comp rx=0 tx=0).stop 2024-02-14T18:20:48.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63a7330640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f63a01986d0 0x7f63a01a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:48.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63a7330640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63a0105380 0x7f63a0198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:48.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63a7330640 1 --2- 172.21.15.100:0/2432772453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f63a01002b0 0x7f63a0197c00 unknown :-1 s=CLOSED pgs=3123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:20:48.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.408+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 >> 172.21.15.100:0/2432772453 conn(0x7f63a00771a0 msgr2=0x7f63a00fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:20:48.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.409+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 shutdown_connections 2024-02-14T18:20:48.410 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:20:48.410+0000 7f63a7330640 1 -- 172.21.15.100:0/2432772453 wait complete. 2024-02-14T18:20:48.426 INFO:tasks.workunit.client.0.smithi100.stderr:+ sleep 30 2024-02-14T18:20:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:20:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:48 smithi179 ceph-mon[33633]: osdmap e792: 8 total, 8 up, 8 in 2024-02-14T18:20:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:48 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:20:48.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:48 smithi179 ceph-mon[33633]: pgmap v1778: 188 pgs: 188 active+clean; 870 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-14T18:20:48.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:20:48.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:48 smithi100 ceph-mon[39557]: osdmap e792: 8 total, 8 up, 8 in 2024-02-14T18:20:48.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:48 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:20:48.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:48 smithi100 ceph-mon[39557]: pgmap v1778: 188 pgs: 188 active+clean; 870 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-14T18:20:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:20:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:48 smithi100 ceph-mon[30138]: osdmap e792: 8 total, 8 up, 8 in 2024-02-14T18:20:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:48 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]: dispatch 2024-02-14T18:20:48.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:48 smithi100 ceph-mon[30138]: pgmap v1778: 188 pgs: 188 active+clean; 870 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-14T18:20:49.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:49 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:20:49.823 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:49 smithi179 ceph-mon[33633]: osdmap e793: 8 total, 8 up, 8 in 2024-02-14T18:20:49.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:49 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:20:49.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:49 smithi100 ceph-mon[30138]: osdmap e793: 8 total, 8 up, 8 in 2024-02-14T18:20:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:49 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/2432772453' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "field": "max_objects", "val": "0"}]': finished 2024-02-14T18:20:49.900 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:49 smithi100 ceph-mon[39557]: osdmap e793: 8 total, 8 up, 8 in 2024-02-14T18:20:50.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:50 smithi179 ceph-mon[33633]: pgmap v1780: 188 pgs: 188 active+clean; 870 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-14T18:20:50.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:50 smithi100 ceph-mon[30138]: pgmap v1780: 188 pgs: 188 active+clean; 870 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-14T18:20:50.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:50 smithi100 ceph-mon[39557]: pgmap v1780: 188 pgs: 188 active+clean; 870 KiB data, 888 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-14T18:20:50.900 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:20:50 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:20:50] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:20:52.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:52 smithi100 ceph-mon[39557]: pgmap v1781: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:52.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:52 smithi100 ceph-mon[30138]: pgmap v1781: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:52.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:52 smithi179 ceph-mon[33633]: pgmap v1781: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:20:53.723 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:20:53 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:20:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:54 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:54.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:54 smithi100 ceph-mon[39557]: pgmap v1782: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 648 B/s rd, 0 op/s 2024-02-14T18:20:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:54 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:54.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:54 smithi100 ceph-mon[30138]: pgmap v1782: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 648 B/s rd, 0 op/s 2024-02-14T18:20:54.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:54 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:20:54.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:54 smithi179 ceph-mon[33633]: pgmap v1782: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 648 B/s rd, 0 op/s 2024-02-14T18:20:56.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:56 smithi100 ceph-mon[39557]: pgmap v1783: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T18:20:56.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:56 smithi100 ceph-mon[30138]: pgmap v1783: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T18:20:56.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:56 smithi179 ceph-mon[33633]: pgmap v1783: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-14T18:20:58.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:58 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:58.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:58 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:58.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:58 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:20:59.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:20:59 smithi179 ceph-mon[33633]: pgmap v1784: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:20:59.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:20:59 smithi100 ceph-mon[39557]: pgmap v1784: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:20:59.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:20:59 smithi100 ceph-mon[30138]: pgmap v1784: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:21:00.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:00 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:21:00] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:21:01.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:01 smithi179 ceph-mon[33633]: pgmap v1785: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-02-14T18:21:01.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:01 smithi100 ceph-mon[39557]: pgmap v1785: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-02-14T18:21:01.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:01 smithi100 ceph-mon[30138]: pgmap v1785: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-02-14T18:21:03.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:03 smithi179 ceph-mon[33633]: pgmap v1786: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:03.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:21:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:21:03.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:03 smithi100 ceph-mon[39557]: pgmap v1786: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:03.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:03 smithi100 ceph-mon[30138]: pgmap v1786: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:04.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:04 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:21:04.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:04 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:21:04.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:04 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-14T18:21:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:05 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:21:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:05 smithi179 ceph-mon[33633]: pgmap v1787: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:05 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:21:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:05 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:21:05.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:05 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:21:05.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:05 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:21:05.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:05 smithi179 ceph-mon[33633]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:21:05.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:21:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[30138]: pgmap v1787: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:21:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:21:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:21:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:21:05.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[30138]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:21:05.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:21:05.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[39557]: pgmap v1787: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:05.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:21:05.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:21:05.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-14T18:21:05.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-14T18:21:05.651 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:05 smithi100 ceph-mon[39557]: from='mgr.24482 ' entity='mgr.y' 2024-02-14T18:21:07.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:07 smithi179 ceph-mon[33633]: pgmap v1788: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:07.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:07 smithi100 ceph-mon[30138]: pgmap v1788: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:07.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:07 smithi100 ceph-mon[39557]: pgmap v1788: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:09.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:09 smithi179 ceph-mon[33633]: pgmap v1789: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:09.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:09 smithi100 ceph-mon[30138]: pgmap v1789: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:09.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:09 smithi100 ceph-mon[39557]: pgmap v1789: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:10.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:10 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:21:10] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:21:11.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:11 smithi179 ceph-mon[33633]: pgmap v1790: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:11.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:11 smithi100 ceph-mon[30138]: pgmap v1790: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:11.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:11 smithi100 ceph-mon[39557]: pgmap v1790: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:13.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:21:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:21:13.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:13 smithi179 ceph-mon[33633]: pgmap v1791: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:13.572 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:13 smithi179 ceph-mon[33633]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:21:13.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:13 smithi100 ceph-mon[30138]: pgmap v1791: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:13.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:13 smithi100 ceph-mon[30138]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:21:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:13 smithi100 ceph-mon[39557]: pgmap v1791: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:13.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:13 smithi100 ceph-mon[39557]: from='mgr.24482 172.21.15.100:0/1469028742' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-14T18:21:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:15 smithi179 ceph-mon[33633]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:21:15.571 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:15 smithi179 ceph-mon[33633]: pgmap v1792: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:15 smithi100 ceph-mon[39557]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:21:15.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:15 smithi100 ceph-mon[39557]: pgmap v1792: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:15 smithi100 ceph-mon[30138]: from='client.24400 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-14T18:21:15.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:15 smithi100 ceph-mon[30138]: pgmap v1792: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:17.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:17 smithi100 ceph-mon[30138]: pgmap v1793: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:17.650 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:17 smithi100 ceph-mon[39557]: pgmap v1793: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:17.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:17 smithi179 ceph-mon[33633]: pgmap v1793: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-14T18:21:18.428 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool delete 8892ea91-15b7-4906-9324-dc08ab1d48a5 8892ea91-15b7-4906-9324-dc08ab1d48a5 --yes-i-really-really-mean-it 2024-02-14T18:21:18.523 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.519+0000 7f150ad3e640 1 -- 172.21.15.100:0/2112294979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041053a0 msgr2=0x7f150410cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.519+0000 7f150ad3e640 1 --2- 172.21.15.100:0/2112294979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041053a0 0x7f150410cf70 secure :-1 s=READY pgs=3124 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009cd0 tx=0x7f14fc02c0e0 comp rx=0 tx=0).stop 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.519+0000 7f150ad3e640 1 -- 172.21.15.100:0/2112294979 shutdown_connections 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.519+0000 7f150ad3e640 1 --2- 172.21.15.100:0/2112294979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041053a0 0x7f150410cf70 unknown :-1 s=CLOSED pgs=3124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.519+0000 7f150ad3e640 1 --2- 172.21.15.100:0/2112294979 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f1504100c80 0x7f1504104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.519+0000 7f150ad3e640 1 --2- 172.21.15.100:0/2112294979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15041002d0 0x7f15041006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.519+0000 7f150ad3e640 1 -- 172.21.15.100:0/2112294979 >> 172.21.15.100:0/2112294979 conn(0x7f1504077260 msgr2=0x7f1504077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.519+0000 7f150ad3e640 1 -- 172.21.15.100:0/2112294979 shutdown_connections 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.519+0000 7f150ad3e640 1 -- 172.21.15.100:0/2112294979 wait complete. 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f150ad3e640 1 Processor -- start 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f150ad3e640 1 -- start start 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f150ad3e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041002d0 0x7f1504196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:21:18.524 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f150ad3e640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15041053a0 0x7f15041972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f150ad3e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f150419b640 0x7f150419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f150ad3e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150419c030 con 0x7f15041002d0 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f150ad3e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150419c1a0 con 0x7f150419b640 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f150ad3e640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150419c310 con 0x7f15041053a0 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f1508ada640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041002d0 0x7f1504196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f1508ada640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041002d0 0x7f1504196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46630/0 (socket says 172.21.15.100:46630) 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f1508ada640 1 -- 172.21.15.100:0/1812832927 learned_addr learned my addr 172.21.15.100:0/1812832927 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f1503fff640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15041053a0 0x7f15041972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f15092db640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f150419b640 0x7f150419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f1508ada640 1 -- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15041053a0 msgr2=0x7f15041972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.520+0000 7f1508ada640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15041053a0 0x7f15041972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.521+0000 7f1508ada640 1 -- 172.21.15.100:0/1812832927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f150419b640 msgr2=0x7f150419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.521+0000 7f1508ada640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f150419b640 0x7f150419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:18.525 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.521+0000 7f1508ada640 1 -- 172.21.15.100:0/1812832927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0009490 con 0x7f15041002d0 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.521+0000 7f1503fff640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15041053a0 0x7f15041972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.521+0000 7f1508ada640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041002d0 0x7f1504196d80 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7f14f800d950 tx=0x7f14f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.521+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8009c30 con 0x7f15041002d0 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.521+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f14f8009d90 con 0x7f15041002d0 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.521+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14fc009900 con 0x7f15041002d0 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.521+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800f620 con 0x7f15041002d0 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.521+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15041a3c10 con 0x7f15041002d0 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.522+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15040684b0 con 0x7f15041002d0 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.522+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7f14f8010460 con 0x7f15041002d0 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.523+0000 7f1501ffb640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f14f406da30 0x7f14f406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.523+0000 7f1503fff640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f14f406da30 0x7f14f406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:21:18.526 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.523+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(793..793 src has 254..793) v4 ==== 9435+0+0 (secure 0 0 0) 0x7f14f808dd30 con 0x7f15041002d0 2024-02-14T18:21:18.527 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.523+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=794}) v3 -- 0x7f14f4079cc0 con 0x7f15041002d0 2024-02-14T18:21:18.527 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.523+0000 7f1503fff640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f14f406da30 0x7f14f406fef0 secure :-1 s=READY pgs=4117 cs=0 l=1 rev1=1 crypto rx=0x7f1504105070 tx=0x7f14f0005f90 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:21:18.528 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.528+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f14f8056a20 con 0x7f15041002d0 2024-02-14T18:21:18.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:18 smithi100 ceph-mon[39557]: pgmap v1794: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:18.649 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:18 smithi100 ceph-mon[30138]: pgmap v1794: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:18.705 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:18.704+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1504100c80 con 0x7f15041002d0 2024-02-14T18:21:18.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:18 smithi179 ceph-mon[33633]: pgmap v1794: 188 pgs: 188 active+clean; 870 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-14T18:21:19.379 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.379+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]=0 pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' removed v794) v1 ==== 248+0+0 (secure 0 0 0) 0x7f14f8020070 con 0x7f15041002d0 2024-02-14T18:21:19.393 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.392+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 <== mon.0 v2:172.21.15.100:3300/0 8 ==== osd_map(794..794 src has 254..794) v4 ==== 296+0+0 (secure 0 0 0) 0x7f14f805e820 con 0x7f15041002d0 2024-02-14T18:21:19.393 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.392+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=795}) v3 -- 0x7f14f407a9a0 con 0x7f15041002d0 2024-02-14T18:21:19.492 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.491+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f150410f650 con 0x7f15041002d0 2024-02-14T18:21:19.493 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.492+0000 7f1501ffb640 1 -- 172.21.15.100:0/1812832927 <== mon.0 v2:172.21.15.100:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]=0 pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' does not exist v794) v1 ==== 255+0+0 (secure 0 0 0) 0x7f14f8017030 con 0x7f15041002d0 2024-02-14T18:21:19.493 INFO:tasks.workunit.client.0.smithi100.stderr:pool '8892ea91-15b7-4906-9324-dc08ab1d48a5' does not exist 2024-02-14T18:21:19.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f14f406da30 msgr2=0x7f14f406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:21:19.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f14f406da30 0x7f14f406fef0 secure :-1 s=READY pgs=4117 cs=0 l=1 rev1=1 crypto rx=0x7f1504105070 tx=0x7f14f0005f90 comp rx=0 tx=0).stop 2024-02-14T18:21:19.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041002d0 msgr2=0x7f1504196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:21:19.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041002d0 0x7f1504196d80 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7f14f800d950 tx=0x7f14f800de10 comp rx=0 tx=0).stop 2024-02-14T18:21:19.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 shutdown_connections 2024-02-14T18:21:19.495 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7f14f406da30 0x7f14f406fef0 unknown :-1 s=CLOSED pgs=4117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:19.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f150419b640 0x7f150419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:19.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7f15041053a0 0x7f15041972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:19.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 --2- 172.21.15.100:0/1812832927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15041002d0 0x7f1504196d80 unknown :-1 s=CLOSED pgs=3125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:19.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 >> 172.21.15.100:0/1812832927 conn(0x7f1504077260 msgr2=0x7f1504104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:21:19.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 shutdown_connections 2024-02-14T18:21:19.496 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.494+0000 7f150ad3e640 1 -- 172.21.15.100:0/1812832927 wait complete. 2024-02-14T18:21:19.504 INFO:tasks.workunit.client.0.smithi100.stderr:+ ceph osd pool delete 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d 36e0b1f1-ec9e-41a3-bde9-02e94ccf390d --yes-i-really-really-mean-it 2024-02-14T18:21:19.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.585+0000 7fb850de9640 1 -- 172.21.15.100:0/2530826389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84c105380 msgr2=0x7fb84c10cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:21:19.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.585+0000 7fb850de9640 1 --2- 172.21.15.100:0/2530826389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84c105380 0x7fb84c10cf50 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7fb844009930 tx=0x7fb84402bc30 comp rx=0 tx=0).stop 2024-02-14T18:21:19.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.585+0000 7fb850de9640 1 -- 172.21.15.100:0/2530826389 shutdown_connections 2024-02-14T18:21:19.590 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.585+0000 7fb850de9640 1 --2- 172.21.15.100:0/2530826389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84c105380 0x7fb84c10cf50 unknown :-1 s=CLOSED pgs=3126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:19.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.585+0000 7fb850de9640 1 --2- 172.21.15.100:0/2530826389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84c100c60 0x7fb84c104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:19.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.585+0000 7fb850de9640 1 --2- 172.21.15.100:0/2530826389 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84c1002b0 0x7fb84c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:19.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.585+0000 7fb850de9640 1 -- 172.21.15.100:0/2530826389 >> 172.21.15.100:0/2530826389 conn(0x7fb84c0771a0 msgr2=0x7fb84c0775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:21:19.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.585+0000 7fb850de9640 1 -- 172.21.15.100:0/2530826389 shutdown_connections 2024-02-14T18:21:19.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.585+0000 7fb850de9640 1 -- 172.21.15.100:0/2530826389 wait complete. 2024-02-14T18:21:19.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.586+0000 7fb850de9640 1 Processor -- start 2024-02-14T18:21:19.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.586+0000 7fb850de9640 1 -- start start 2024-02-14T18:21:19.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.586+0000 7fb850de9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84c1002b0 0x7fb84c10d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:21:19.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.586+0000 7fb850de9640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84c105380 0x7fb84c10dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:21:19.591 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.586+0000 7fb850de9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84c10e2b0 0x7fb84c1a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:21:19.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.586+0000 7fb850de9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c1a5430 con 0x7fb84c10e2b0 2024-02-14T18:21:19.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.586+0000 7fb850de9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c1a55a0 con 0x7fb84c1002b0 2024-02-14T18:21:19.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.586+0000 7fb850de9640 1 -- --> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c1a5760 con 0x7fb84c105380 2024-02-14T18:21:19.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.586+0000 7fb849d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84c105380 0x7fb84c10dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:21:19.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.586+0000 7fb84ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84c10e2b0 0x7fb84c1a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:21:19.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb84ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84c10e2b0 0x7fb84c1a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46646/0 (socket says 172.21.15.100:46646) 2024-02-14T18:21:19.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb84ad9d640 1 -- 172.21.15.100:0/1746379950 learned_addr learned my addr 172.21.15.100:0/1746379950 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-02-14T18:21:19.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb849d9b640 1 --2- >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84c105380 0x7fb84c10dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3301/0 says I am v2:172.21.15.100:43176/0 (socket says 172.21.15.100:43176) 2024-02-14T18:21:19.592 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb84a59c640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84c1002b0 0x7fb84c10d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:21:19.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb84ad9d640 1 -- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84c105380 msgr2=0x7fb84c10dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:21:19.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb84ad9d640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84c105380 0x7fb84c10dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:19.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb84ad9d640 1 -- 172.21.15.100:0/1746379950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84c1002b0 msgr2=0x7fb84c10d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:21:19.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb84ad9d640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84c1002b0 0x7fb84c10d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:19.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb84ad9d640 1 -- 172.21.15.100:0/1746379950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c009840 con 0x7fb84c10e2b0 2024-02-14T18:21:19.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb84a59c640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84c1002b0 0x7fb84c10d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-14T18:21:19.593 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb849d9b640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84c105380 0x7fb84c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb84ad9d640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84c10e2b0 0x7fb84c1a4e60 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7fb84402c6b0 tx=0x7fb8440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844004aa0 con 0x7fb84c10e2b0 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb8440044e0 con 0x7fb84c10e2b0 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb844009560 con 0x7fb84c10e2b0 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8440052f0 con 0x7fb84c10e2b0 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.587+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c1a5db0 con 0x7fb84c10e2b0 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.588+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84c068060 con 0x7fb84c10e2b0 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.588+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 22) v1 ==== 91465+0+0 (secure 0 0 0) 0x7fb844005450 con 0x7fb84c10e2b0 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.589+0000 7fb83b7fe640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb83406da60 0x7fb83406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.589+0000 7fb84a59c640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb83406da60 0x7fb83406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-14T18:21:19.594 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.589+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(794..794 src has 254..794) v4 ==== 9047+0+0 (secure 0 0 0) 0x7fb8440b01a0 con 0x7fb84c10e2b0 2024-02-14T18:21:19.595 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.589+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=795}) v3 -- 0x7fb834079550 con 0x7fb84c10e2b0 2024-02-14T18:21:19.595 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.589+0000 7fb84a59c640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb83406da60 0x7fb83406ff20 secure :-1 s=READY pgs=4118 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005fd0 tx=0x7fb83c009500 comp rx=0 tx=0).ready entity=mgr.24482 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-14T18:21:19.595 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.594+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb844078fe0 con 0x7fb84c10e2b0 2024-02-14T18:21:19.649 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:19 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1812832927' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:19.650 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:19 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1812832927' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:19.769 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:19.768+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb84c108610 con 0x7fb84c10e2b0 2024-02-14T18:21:19.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:19 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1812832927' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:20.385 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.384+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]=0 pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' removed v795) v1 ==== 248+0+0 (secure 0 0 0) 0x7fb844043100 con 0x7fb84c10e2b0 2024-02-14T18:21:20.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.385+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 <== mon.0 v2:172.21.15.100:3300/0 8 ==== osd_map(795..795 src has 254..795) v4 ==== 296+0+0 (secure 0 0 0) 0x7fb84407d440 con 0x7fb84c10e2b0 2024-02-14T18:21:20.386 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.385+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=796}) v3 -- 0x7fb83407a1f0 con 0x7fb84c10e2b0 2024-02-14T18:21:20.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.500+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb84c100e30 con 0x7fb84c10e2b0 2024-02-14T18:21:20.501 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.500+0000 7fb83b7fe640 1 -- 172.21.15.100:0/1746379950 <== mon.0 v2:172.21.15.100:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]=0 pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' does not exist v795) v1 ==== 255+0+0 (secure 0 0 0) 0x7fb84407c6a0 con 0x7fb84c10e2b0 2024-02-14T18:21:20.501 INFO:tasks.workunit.client.0.smithi100.stderr:pool '36e0b1f1-ec9e-41a3-bde9-02e94ccf390d' does not exist 2024-02-14T18:21:20.503 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.502+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb83406da60 msgr2=0x7fb83406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.502+0000 7fb850de9640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb83406da60 0x7fb83406ff20 secure :-1 s=READY pgs=4118 cs=0 l=1 rev1=1 crypto rx=0x7fb83c005fd0 tx=0x7fb83c009500 comp rx=0 tx=0).stop 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.502+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84c10e2b0 msgr2=0x7fb84c1a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.502+0000 7fb850de9640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84c10e2b0 0x7fb84c1a4e60 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7fb84402c6b0 tx=0x7fb8440042f0 comp rx=0 tx=0).stop 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.502+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 shutdown_connections 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.502+0000 7fb850de9640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:6800/1773294710,v1:172.21.15.100:6801/1773294710] conn(0x7fb83406da60 0x7fb83406ff20 unknown :-1 s=CLOSED pgs=4118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.502+0000 7fb850de9640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb84c10e2b0 0x7fb84c1a4e60 unknown :-1 s=CLOSED pgs=3127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.502+0000 7fb850de9640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.100:3301/0,v1:172.21.15.100:6790/0] conn(0x7fb84c105380 0x7fb84c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.502+0000 7fb850de9640 1 --2- 172.21.15.100:0/1746379950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb84c1002b0 0x7fb84c10d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.503+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 >> 172.21.15.100:0/1746379950 conn(0x7fb84c0771a0 msgr2=0x7fb84c106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.503+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 shutdown_connections 2024-02-14T18:21:20.504 INFO:tasks.workunit.client.0.smithi100.stderr:2024-02-14T18:21:20.503+0000 7fb850de9640 1 -- 172.21.15.100:0/1746379950 wait complete. 2024-02-14T18:21:20.513 INFO:tasks.workunit.client.0.smithi100.stdout:OK 2024-02-14T18:21:20.514 INFO:tasks.workunit.client.0.smithi100.stderr:+ echo OK 2024-02-14T18:21:20.514 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-14T18:21:20.514 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-14T18:21:20.583 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-02-14T18:21:20.583 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-14T18:21:20.646 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: ::ffff:172.21.15.179 - - [14/Feb/2024:18:21:20] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-14T18:21:20.647 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1812832927' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:21:20.647 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-mon[39557]: osdmap e794: 8 total, 8 up, 8 in 2024-02-14T18:21:20.647 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1812832927' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:20.647 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1746379950' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:20.647 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-mon[39557]: pgmap v1796: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:21:20.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1812832927' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:21:20.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-mon[30138]: osdmap e794: 8 total, 8 up, 8 in 2024-02-14T18:21:20.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1812832927' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:20.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1746379950' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:20.647 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:20 smithi100 ceph-mon[30138]: pgmap v1796: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:21:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1812832927' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:21:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:20 smithi179 ceph-mon[33633]: osdmap e794: 8 total, 8 up, 8 in 2024-02-14T18:21:20.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1812832927' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "pool2": "8892ea91-15b7-4906-9324-dc08ab1d48a5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:20.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:20 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1746379950' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:20.822 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:20 smithi179 ceph-mon[33633]: pgmap v1796: 176 pgs: 176 active+clean; 697 KiB data, 889 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-14T18:21:20.887 DEBUG:teuthology.parallel:result is None 2024-02-14T18:21:20.888 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-14T18:21:20.955 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-14T18:21:20.955 DEBUG:teuthology.orchestra.run.smithi100:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-14T18:21:21.020 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-14T18:21:21.021 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-14T18:21:21.031 INFO:tasks.cephadm:Teardown begin 2024-02-14T18:21:21.032 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T18:21:21.087 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T18:21:21.122 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-14T18:21:21.122 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-14T18:21:21.143 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-14T18:21:21.179 INFO:tasks.cephadm:Stopping all daemons... 2024-02-14T18:21:21.179 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-14T18:21:21.180 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.a 2024-02-14T18:21:21.478 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:21 smithi100 ceph-mon[39557]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:21:21.478 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1746379950' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:21:21.478 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:21 smithi100 ceph-mon[39557]: osdmap e795: 8 total, 8 up, 8 in 2024-02-14T18:21:21.478 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:21 smithi100 ceph-mon[39557]: from='client.? 172.21.15.100:0/1746379950' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:21.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:21 smithi100 systemd[1]: Stopping Ceph mon.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:21:21.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:21 smithi100 ceph-mon[30138]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:21:21.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1746379950' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:21:21.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:21 smithi100 ceph-mon[30138]: osdmap e795: 8 total, 8 up, 8 in 2024-02-14T18:21:21.479 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:21 smithi100 ceph-mon[30138]: from='client.? 172.21.15.100:0/1746379950' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:21 smithi179 ceph-mon[33633]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-14T18:21:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1746379950' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]': finished 2024-02-14T18:21:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:21 smithi179 ceph-mon[33633]: osdmap e795: 8 total, 8 up, 8 in 2024-02-14T18:21:21.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:21 smithi179 ceph-mon[33633]: from='client.? 172.21.15.100:0/1746379950' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "pool2": "36e0b1f1-ec9e-41a3-bde9-02e94ccf390d", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-14T18:21:21.899 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:21 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a[30115]: 2024-02-14T18:21:21.477+0000 7efcbcc15700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:21:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:21 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a[30115]: 2024-02-14T18:21:21.477+0000 7efcbcc15700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-14T18:21:21.900 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:21 smithi100 podman[158038]: 2024-02-14 18:21:21.651363417 +0000 UTC m=+0.285127788 container died 79efe7e1a03d9550542f6899f0b3c280f92592f974c853059781ba8a2c9e0883 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-14T18:21:22.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:22 smithi100 podman[158038]: 2024-02-14 18:21:22.158991355 +0000 UTC m=+0.792755733 container cleanup 79efe7e1a03d9550542f6899f0b3c280f92592f974c853059781ba8a2c9e0883 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:21:22.327 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:22 smithi100 bash[158038]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a 2024-02-14T18:21:22.574 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.a.service' 2024-02-14T18:21:22.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:22 smithi100 podman[158057]: 2024-02-14 18:21:22.327066184 +0000 UTC m=+0.674728918 container remove 79efe7e1a03d9550542f6899f0b3c280f92592f974c853059781ba8a2c9e0883 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-a, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-14T18:21:22.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:22 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-14T18:21:22.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:22 smithi100 systemd[1]: Stopped Ceph mon.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:21:22.598 INFO:journalctl@ceph.mon.a.smithi100.stdout:Feb 14 18:21:22 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.a.service: Consumed 2min 10.094s CPU time. 2024-02-14T18:21:22.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:21:22.614 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-14T18:21:22.614 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-14T18:21:22.614 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.c 2024-02-14T18:21:22.899 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:22 smithi100 systemd[1]: Stopping Ceph mon.c for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:21:23.399 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:22 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-c[39533]: 2024-02-14T18:21:22.973+0000 7f0901320700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:21:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:22 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-c[39533]: 2024-02-14T18:21:22.973+0000 7f0901320700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-14T18:21:23.400 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:23 smithi100 podman[158170]: 2024-02-14 18:21:23.178625388 +0000 UTC m=+0.343205849 container died e5501fd1d64a369bf51fb0e4b5b322da7bf7cdba97439067fba6f43628c293f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-14T18:21:23.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:22 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:18:21:22] ENGINE Bus STOPPING 2024-02-14T18:21:23.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:22 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:18:21:22] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-14T18:21:23.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:22 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:18:21:22] ENGINE Bus STOPPED 2024-02-14T18:21:23.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:22 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:18:21:22] ENGINE Bus STARTING 2024-02-14T18:21:23.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:23 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:18:21:23] ENGINE Serving on http://:::9283 2024-02-14T18:21:23.400 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:23 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: [14/Feb/2024:18:21:23] ENGINE Bus STARTED 2024-02-14T18:21:23.806 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:23 smithi100 podman[158170]: 2024-02-14 18:21:23.622089814 +0000 UTC m=+0.786670285 container cleanup e5501fd1d64a369bf51fb0e4b5b322da7bf7cdba97439067fba6f43628c293f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-c, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0) 2024-02-14T18:21:23.806 INFO:journalctl@ceph.mon.c.smithi100.stdout:Feb 14 18:21:23 smithi100 bash[158170]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-c 2024-02-14T18:21:23.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:21:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug there is no tcmu-runner data available 2024-02-14T18:21:24.012 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.c.service' 2024-02-14T18:21:24.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:21:24.050 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-14T18:21:24.050 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-14T18:21:24.050 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.b 2024-02-14T18:21:24.401 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:24 smithi179 systemd[1]: Stopping Ceph mon.b for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:21:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-b[33596]: 2024-02-14T18:21:24.399+0000 7f8810764700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:21:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-b[33596]: 2024-02-14T18:21:24.399+0000 7f8810764700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-14T18:21:24.821 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:24 smithi179 podman[78758]: 2024-02-14 18:21:24.645096034 +0000 UTC m=+0.394859070 container died c6063f9d54e1c0525551323259469b5cdf481ec5c1634c633c8e165b5a334ef0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-b, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.build-date=20240214) 2024-02-14T18:21:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:25 smithi179 podman[78758]: 2024-02-14 18:21:25.098360788 +0000 UTC m=+0.848123874 container cleanup c6063f9d54e1c0525551323259469b5cdf481ec5c1634c633c8e165b5a334ef0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-b, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-14T18:21:25.321 INFO:journalctl@ceph.mon.b.smithi179.stdout:Feb 14 18:21:25 smithi179 bash[78758]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mon-b 2024-02-14T18:21:25.516 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mon.b.service' 2024-02-14T18:21:25.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:21:25.569 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-14T18:21:25.569 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-14T18:21:25.569 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.y 2024-02-14T18:21:25.899 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:25 smithi100 systemd[1]: Stopping Ceph mgr.y for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:21:26.352 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:26 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T18:21:26.020+0000 7efc5a5b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:21:26.353 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:26 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y[30391]: 2024-02-14T18:21:26.020+0000 7efc5a5b6700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-14T18:21:26.353 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:26 smithi100 podman[158313]: 2024-02-14 18:21:26.052495708 +0000 UTC m=+0.297641875 container died c60fabca0e5e6273da781a37e120198f3f7eba0c92a152afc21e861929b2335f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-14T18:21:26.611 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:26 smithi100 podman[158313]: 2024-02-14 18:21:26.443550164 +0000 UTC m=+0.688696312 container cleanup c60fabca0e5e6273da781a37e120198f3f7eba0c92a152afc21e861929b2335f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T18:21:26.612 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:26 smithi100 bash[158313]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y 2024-02-14T18:21:26.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.y.service' 2024-02-14T18:21:26.882 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:26 smithi100 podman[158330]: 2024-02-14 18:21:26.611570212 +0000 UTC m=+0.557803284 container remove c60fabca0e5e6273da781a37e120198f3f7eba0c92a152afc21e861929b2335f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8727ce939fc044a98f215e29696bb1433f5758d1, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-y, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-14T18:21:26.882 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:26 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.y.service: Deactivated successfully. 2024-02-14T18:21:26.882 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:26 smithi100 systemd[1]: Stopped Ceph mgr.y for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:21:26.882 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Feb 14 18:21:26 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.y.service: Consumed 2min 6.855s CPU time. 2024-02-14T18:21:26.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:21:26.896 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-14T18:21:26.896 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-14T18:21:26.897 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.x 2024-02-14T18:21:27.237 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 18:21:26 smithi179 systemd[1]: Stopping Ceph mgr.x for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:21:27.513 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 18:21:27 smithi179 podman[78887]: 2024-02-14 18:21:27.236354119 +0000 UTC m=+0.153590330 container died 3f087bfacba1634488f736d88895c59dd3f558925e1c5c9defd83494a42b03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-14T18:21:27.821 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 18:21:27 smithi179 podman[78887]: 2024-02-14 18:21:27.585789936 +0000 UTC m=+0.503026152 container cleanup 3f087bfacba1634488f736d88895c59dd3f558925e1c5c9defd83494a42b03ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-14T18:21:27.821 INFO:journalctl@ceph.mgr.x.smithi179.stdout:Feb 14 18:21:27 smithi179 bash[78887]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-mgr-x 2024-02-14T18:21:28.012 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@mgr.x.service' 2024-02-14T18:21:28.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:21:28.050 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-14T18:21:28.050 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-14T18:21:28.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.0 2024-02-14T18:21:28.399 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:28 smithi100 systemd[1]: Stopping Ceph osd.0 for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:21:28.900 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:28 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0[43959]: 2024-02-14T18:21:28.466+0000 7f813bc25700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:21:28.900 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:28 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0[43959]: 2024-02-14T18:21:28.466+0000 7f813bc25700 -1 osd.0 795 *** Got signal Terminated *** 2024-02-14T18:21:28.900 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:28 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0[43959]: 2024-02-14T18:21:28.466+0000 7f813bc25700 -1 osd.0 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:21:34.307 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:33 smithi100 podman[158443]: 2024-02-14 18:21:33.981384764 +0000 UTC m=+5.736060563 container died 9da65abc09cb9bf67c7ee048c7db430aa897e822272d674aee8bd56840b320d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux ) 2024-02-14T18:21:34.624 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:34 smithi100 podman[158443]: 2024-02-14 18:21:34.389136923 +0000 UTC m=+6.143812721 container cleanup 9da65abc09cb9bf67c7ee048c7db430aa897e822272d674aee8bd56840b320d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-02-14T18:21:34.624 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:34 smithi100 bash[158443]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0 2024-02-14T18:21:34.899 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:34 smithi100 podman[158463]: 2024-02-14 18:21:34.624003247 +0000 UTC m=+0.642388305 container remove 9da65abc09cb9bf67c7ee048c7db430aa897e822272d674aee8bd56840b320d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-14T18:21:35.269 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:34 smithi100 podman[158561]: 2024-02-14 18:21:34.848990949 +0000 UTC m=+0.018356529 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:21:35.269 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:34 smithi100 podman[158561]: 2024-02-14 18:21:34.983254481 +0000 UTC m=+0.152620058 container create 7daf2e162f1087366b82320379e482227826e9849f5cb2eeeccb0d930fd7fea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-14T18:21:35.269 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:35 smithi100 podman[158561]: 2024-02-14 18:21:35.185659672 +0000 UTC m=+0.355025249 container init 7daf2e162f1087366b82320379e482227826e9849f5cb2eeeccb0d930fd7fea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-14T18:21:35.269 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:35 smithi100 podman[158561]: 2024-02-14 18:21:35.191427349 +0000 UTC m=+0.360792930 container start 7daf2e162f1087366b82320379e482227826e9849f5cb2eeeccb0d930fd7fea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0-deactivate, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-14T18:21:35.269 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:35 smithi100 podman[158561]: 2024-02-14 18:21:35.268823405 +0000 UTC m=+0.438188982 container attach 7daf2e162f1087366b82320379e482227826e9849f5cb2eeeccb0d930fd7fea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-14T18:21:35.649 INFO:journalctl@ceph.osd.0.smithi100.stdout:Feb 14 18:21:35 smithi100 podman[158561]: 2024-02-14 18:21:35.48427481 +0000 UTC m=+0.653640396 container died 7daf2e162f1087366b82320379e482227826e9849f5cb2eeeccb0d930fd7fea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-14T18:21:36.066 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.0.service' 2024-02-14T18:21:36.113 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:21:36.113 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-14T18:21:36.113 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-14T18:21:36.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.1 2024-02-14T18:21:36.400 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:36 smithi100 systemd[1]: Stopping Ceph osd.1 for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:21:36.900 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:36 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1[49905]: 2024-02-14T18:21:36.410+0000 7ff690c54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:21:36.900 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:36 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1[49905]: 2024-02-14T18:21:36.410+0000 7ff690c54700 -1 osd.1 795 *** Got signal Terminated *** 2024-02-14T18:21:36.900 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:36 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1[49905]: 2024-02-14T18:21:36.410+0000 7ff690c54700 -1 osd.1 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:21:42.252 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:41 smithi100 podman[158696]: 2024-02-14 18:21:41.955081894 +0000 UTC m=+5.658761519 container died d2400115de2849786b64acf222bfa0f18c5dd7290697badd51f05943f09d44f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-14T18:21:42.585 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:42 smithi100 podman[158696]: 2024-02-14 18:21:42.358636736 +0000 UTC m=+6.062316364 container cleanup d2400115de2849786b64acf222bfa0f18c5dd7290697badd51f05943f09d44f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T18:21:42.585 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:42 smithi100 bash[158696]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1 2024-02-14T18:21:42.899 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:42 smithi100 podman[158718]: 2024-02-14 18:21:42.585080873 +0000 UTC m=+0.627205532 container remove d2400115de2849786b64acf222bfa0f18c5dd7290697badd51f05943f09d44f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T18:21:43.169 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:42 smithi100 podman[158815]: 2024-02-14 18:21:42.818604255 +0000 UTC m=+0.017845832 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:21:43.169 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:42 smithi100 podman[158815]: 2024-02-14 18:21:42.927577257 +0000 UTC m=+0.126818862 container create ad7274417f63b2208ad66d6f15109e17316ecefe910f562ff623fc3665f36834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T18:21:43.169 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:43 smithi100 podman[158815]: 2024-02-14 18:21:43.163281019 +0000 UTC m=+0.362522584 container init ad7274417f63b2208ad66d6f15109e17316ecefe910f562ff623fc3665f36834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-14T18:21:43.169 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:43 smithi100 podman[158815]: 2024-02-14 18:21:43.168900236 +0000 UTC m=+0.368141807 container start ad7274417f63b2208ad66d6f15109e17316ecefe910f562ff623fc3665f36834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2) 2024-02-14T18:21:43.459 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:43 smithi100 podman[158815]: 2024-02-14 18:21:43.254770971 +0000 UTC m=+0.454012537 container attach ad7274417f63b2208ad66d6f15109e17316ecefe910f562ff623fc3665f36834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1-deactivate, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T18:21:43.749 INFO:journalctl@ceph.osd.1.smithi100.stdout:Feb 14 18:21:43 smithi100 podman[158815]: 2024-02-14 18:21:43.459558468 +0000 UTC m=+0.658800031 container died ad7274417f63b2208ad66d6f15109e17316ecefe910f562ff623fc3665f36834 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2) 2024-02-14T18:21:44.084 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.1.service' 2024-02-14T18:21:44.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:21:44.126 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-14T18:21:44.126 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-14T18:21:44.126 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.2 2024-02-14T18:21:44.347 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:44 smithi100 systemd[1]: Stopping Ceph osd.2 for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:21:44.649 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2[56161]: 2024-02-14T18:21:44.489+0000 7f0a54304700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:21:44.649 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2[56161]: 2024-02-14T18:21:44.489+0000 7f0a54304700 -1 osd.2 795 *** Got signal Terminated *** 2024-02-14T18:21:44.649 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:44 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2[56161]: 2024-02-14T18:21:44.489+0000 7f0a54304700 -1 osd.2 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:21:50.354 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:50 smithi100 podman[158945]: 2024-02-14 18:21:50.050270617 +0000 UTC m=+5.699221199 container died 5edd961e8fa95a99920c170af21b668b7d19acb7fcbd4ae6a2a7fd07c0a41715 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-14T18:21:50.649 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:50 smithi100 podman[158945]: 2024-02-14 18:21:50.486812953 +0000 UTC m=+6.135763532 container cleanup 5edd961e8fa95a99920c170af21b668b7d19acb7fcbd4ae6a2a7fd07c0a41715 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-14T18:21:50.650 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:50 smithi100 bash[158945]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2 2024-02-14T18:21:51.097 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:50 smithi100 podman[158982]: 2024-02-14 18:21:50.671450705 +0000 UTC m=+0.623654108 container remove 5edd961e8fa95a99920c170af21b668b7d19acb7fcbd4ae6a2a7fd07c0a41715 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, ceph=True) 2024-02-14T18:21:51.399 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:51 smithi100 podman[159077]: 2024-02-14 18:21:51.097427977 +0000 UTC m=+0.112870556 container create 70b45b7656422b88cbc9b15c1e782d75c3012704b449c59ec33dd7668d76ee67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T18:21:51.399 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:51 smithi100 podman[159077]: 2024-02-14 18:21:51.004245482 +0000 UTC m=+0.019688057 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:21:51.400 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:51 smithi100 podman[159077]: 2024-02-14 18:21:51.316424958 +0000 UTC m=+0.331867536 container init 70b45b7656422b88cbc9b15c1e782d75c3012704b449c59ec33dd7668d76ee67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS) 2024-02-14T18:21:51.400 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:51 smithi100 podman[159077]: 2024-02-14 18:21:51.322183016 +0000 UTC m=+0.337625593 container start 70b45b7656422b88cbc9b15c1e782d75c3012704b449c59ec33dd7668d76ee67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-02-14T18:21:51.400 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:51 smithi100 podman[159077]: 2024-02-14 18:21:51.399620258 +0000 UTC m=+0.415062838 container attach 70b45b7656422b88cbc9b15c1e782d75c3012704b449c59ec33dd7668d76ee67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:21:51.899 INFO:journalctl@ceph.osd.2.smithi100.stdout:Feb 14 18:21:51 smithi100 podman[159077]: 2024-02-14 18:21:51.608604147 +0000 UTC m=+0.624046731 container died 70b45b7656422b88cbc9b15c1e782d75c3012704b449c59ec33dd7668d76ee67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-14T18:21:52.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.2.service' 2024-02-14T18:21:52.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:21:52.237 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-14T18:21:52.237 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-14T18:21:52.238 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.3 2024-02-14T18:21:52.567 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:52 smithi100 systemd[1]: Stopping Ceph osd.3 for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:21:52.899 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:52 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3[62804]: 2024-02-14T18:21:52.566+0000 7f58eac0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:21:52.899 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:52 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3[62804]: 2024-02-14T18:21:52.566+0000 7f58eac0e700 -1 osd.3 795 *** Got signal Terminated *** 2024-02-14T18:21:52.899 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:52 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3[62804]: 2024-02-14T18:21:52.566+0000 7f58eac0e700 -1 osd.3 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:21:57.072 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:21:56 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:21:56.727+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.218984+0000 front 2024-02-14T18:21:33.219032+0000 (oldest deadline 2024-02-14T18:21:56.118747+0000) 2024-02-14T18:21:57.344 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:21:57 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:21:57.086+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:21:57.754 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:21:57 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:21:57.421+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:21:58.071 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:21:57 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:21:57.752+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.218984+0000 front 2024-02-14T18:21:33.219032+0000 (oldest deadline 2024-02-14T18:21:56.118747+0000) 2024-02-14T18:21:58.357 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:58 smithi100 podman[159207]: 2024-02-14 18:21:58.068642524 +0000 UTC m=+5.613809152 container died d9e12d130d98a7fd2b3c0058cd7bfa03a977afedcc173a205311362abbd17a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5) 2024-02-14T18:21:58.448 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:21:58 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:21:58.105+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:21:58.649 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:58 smithi100 podman[159207]: 2024-02-14 18:21:58.438376531 +0000 UTC m=+5.983543140 container cleanup d9e12d130d98a7fd2b3c0058cd7bfa03a977afedcc173a205311362abbd17a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-14T18:21:58.649 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:58 smithi100 bash[159207]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3 2024-02-14T18:21:58.753 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:21:58 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:21:58.446+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:21:59.005 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:58 smithi100 podman[159228]: 2024-02-14 18:21:58.689666679 +0000 UTC m=+0.623479474 container remove d9e12d130d98a7fd2b3c0058cd7bfa03a977afedcc173a205311362abbd17a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-14T18:21:59.046 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:21:58 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:21:58.752+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.218984+0000 front 2024-02-14T18:21:33.219032+0000 (oldest deadline 2024-02-14T18:21:56.118747+0000) 2024-02-14T18:21:59.262 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:59 smithi100 podman[159326]: 2024-02-14 18:21:58.909366572 +0000 UTC m=+0.018669543 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:21:59.274 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:59 smithi100 podman[159326]: 2024-02-14 18:21:59.015348218 +0000 UTC m=+0.124651185 container create 102ef89a573da8575691039e053cf607f269c443db647cc5d28e4065611aa464 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214) 2024-02-14T18:21:59.274 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:59 smithi100 podman[159326]: 2024-02-14 18:21:59.262604396 +0000 UTC m=+0.371907363 container init 102ef89a573da8575691039e053cf607f269c443db647cc5d28e4065611aa464 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T18:21:59.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:21:59 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:21:59.099+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:21:59.323 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:21:59 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:21:59.045+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:21:59.554 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:59 smithi100 podman[159326]: 2024-02-14 18:21:59.268003221 +0000 UTC m=+0.377306191 container start 102ef89a573da8575691039e053cf607f269c443db647cc5d28e4065611aa464 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-14T18:21:59.554 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:59 smithi100 podman[159326]: 2024-02-14 18:21:59.367843226 +0000 UTC m=+0.477146221 container attach 102ef89a573da8575691039e053cf607f269c443db647cc5d28e4065611aa464 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:21:59.803 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:21:59 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:21:59.407+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:21:59.845 INFO:journalctl@ceph.osd.3.smithi100.stdout:Feb 14 18:21:59 smithi100 podman[159326]: 2024-02-14 18:21:59.553952397 +0000 UTC m=+0.663255364 container died 102ef89a573da8575691039e053cf607f269c443db647cc5d28e4065611aa464 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-14T18:22:00.071 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:21:59 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:21:59.802+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.218984+0000 front 2024-02-14T18:21:33.219032+0000 (oldest deadline 2024-02-14T18:21:56.118747+0000) 2024-02-14T18:22:00.071 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:00 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:00.002+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:00.132 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.3.service' 2024-02-14T18:22:00.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:22:00.173 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-14T18:22:00.173 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-14T18:22:00.173 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.4 2024-02-14T18:22:00.413 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:00 smithi179 systemd[1]: Stopping Ceph osd.4 for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:22:00.413 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:00 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:00.081+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:00.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:00 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:00.412+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:00.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:00 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:22:00.590+0000 7f078317f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:22:00.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:00 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:22:00.590+0000 7f078317f700 -1 osd.4 795 *** Got signal Terminated *** 2024-02-14T18:22:00.821 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:00 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:22:00.590+0000 7f078317f700 -1 osd.4 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:22:01.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:01.055+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:01.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:01.037+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:01.322 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:00 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:22:00.825+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.218984+0000 front 2024-02-14T18:21:33.219032+0000 (oldest deadline 2024-02-14T18:21:56.118747+0000) 2024-02-14T18:22:01.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:01.398+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:02.321 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:01 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:22:01.854+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.218984+0000 front 2024-02-14T18:21:33.219032+0000 (oldest deadline 2024-02-14T18:21:56.118747+0000) 2024-02-14T18:22:02.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:02.023+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:02.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:02.045+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:02.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:02.421+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:02.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:02.421+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:03.321 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:02 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:22:02.836+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.218984+0000 front 2024-02-14T18:21:33.219032+0000 (oldest deadline 2024-02-14T18:21:56.118747+0000) 2024-02-14T18:22:03.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:03.014+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:03.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:03.014+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:03.322 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:03.037+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:03.322 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:03.037+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:03.812 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:03.417+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:03.812 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:03.417+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:04.071 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:04 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:04.026+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:04.071 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:04 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:04.026+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:04.071 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:22:03.810+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.218984+0000 front 2024-02-14T18:21:33.219032+0000 (oldest deadline 2024-02-14T18:21:56.118747+0000) 2024-02-14T18:22:04.071 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:03 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:22:03.810+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.819723+0000 front 2024-02-14T18:21:40.819679+0000 (oldest deadline 2024-02-14T18:22:03.119740+0000) 2024-02-14T18:22:04.437 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:04 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:04.073+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:04.437 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:04 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:04.073+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:04.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:04 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:04.435+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:04.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:04 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:04.435+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:05.221 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:04 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:22:04.858+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.218984+0000 front 2024-02-14T18:21:33.219032+0000 (oldest deadline 2024-02-14T18:21:56.118747+0000) 2024-02-14T18:22:05.221 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:04 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4[39159]: 2024-02-14T18:22:04.858+0000 7f077b76b700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.819723+0000 front 2024-02-14T18:21:40.819679+0000 (oldest deadline 2024-02-14T18:22:03.119740+0000) 2024-02-14T18:22:05.221 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:05.036+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:05.221 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:05.036+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:05.221 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:05.049+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:05.222 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:05.049+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:05.482 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:05.220Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.100:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.100:8765: connect: connection refused" 2024-02-14T18:22:05.482 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:05.220Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.100:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.100:8765: connect: connection refused" 2024-02-14T18:22:05.482 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:05.221Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.100:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.100:8765: connect: connection refused" 2024-02-14T18:22:05.482 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:05.221Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.100:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.100:8765: connect: connection refused" 2024-02-14T18:22:05.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:05.481+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:05.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:05 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:05.481+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:06.191 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:05 smithi179 podman[79032]: 2024-02-14 18:22:05.928715717 +0000 UTC m=+5.554757556 container died e3f23b5144342a7eee1ee0424d666fd44e0882f6b48dddb5d3bd1a16166bab57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True) 2024-02-14T18:22:06.191 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:06 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:06.057+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:06.191 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:06 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:06.057+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:06.191 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:06 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:06.059+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:06.191 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:06 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:06.059+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:06.455 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:06 smithi179 podman[79032]: 2024-02-14 18:22:06.279729626 +0000 UTC m=+5.905771465 container cleanup e3f23b5144342a7eee1ee0424d666fd44e0882f6b48dddb5d3bd1a16166bab57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-14T18:22:06.455 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:06 smithi179 bash[79032]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4 2024-02-14T18:22:06.822 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:06 smithi179 podman[79054]: 2024-02-14 18:22:06.539332261 +0000 UTC m=+0.611255454 container remove e3f23b5144342a7eee1ee0424d666fd44e0882f6b48dddb5d3bd1a16166bab57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:22:06.823 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:06 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:06.453+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:06.823 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:06 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:06.453+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:07.152 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:07 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:07.085+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:07.152 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:07 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:07.085+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:07.152 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:06 smithi179 podman[79147]: 2024-02-14 18:22:06.798600843 +0000 UTC m=+0.017473730 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:22:07.152 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:06 smithi179 podman[79147]: 2024-02-14 18:22:06.923819484 +0000 UTC m=+0.142692359 container create 7483d481e4686cb56d6b01d578f49adcf43f470d914f8cad1581da85a54656a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T18:22:07.152 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:07 smithi179 podman[79147]: 2024-02-14 18:22:07.142963486 +0000 UTC m=+0.361836377 container init 7483d481e4686cb56d6b01d578f49adcf43f470d914f8cad1581da85a54656a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-14T18:22:07.153 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:07 smithi179 podman[79147]: 2024-02-14 18:22:07.148842952 +0000 UTC m=+0.367715829 container start 7483d481e4686cb56d6b01d578f49adcf43f470d914f8cad1581da85a54656a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-14T18:22:07.153 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:07 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:07.047+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:07.153 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:07 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:07.047+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:07.442 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:07 smithi179 podman[79147]: 2024-02-14 18:22:07.251171577 +0000 UTC m=+0.470044468 container attach 7483d481e4686cb56d6b01d578f49adcf43f470d914f8cad1581da85a54656a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-14T18:22:07.730 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:07 smithi179 podman[79147]: 2024-02-14 18:22:07.441868684 +0000 UTC m=+0.660741563 container died 7483d481e4686cb56d6b01d578f49adcf43f470d914f8cad1581da85a54656a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:22:07.730 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:07 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:07.459+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:07.730 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:07 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:07.459+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:07.993 INFO:journalctl@ceph.osd.4.smithi179.stdout:Feb 14 18:22:07 smithi179 podman[79194]: 2024-02-14 18:22:07.970978519 +0000 UTC m=+0.523183150 container remove 7483d481e4686cb56d6b01d578f49adcf43f470d914f8cad1581da85a54656a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-14T18:22:07.999 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.4.service' 2024-02-14T18:22:08.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:22:08.039 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-14T18:22:08.040 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-14T18:22:08.040 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.5 2024-02-14T18:22:08.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:08.074+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:08.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:08.074+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:08.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:08.004+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:08.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:08.004+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:08.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:08 smithi179 systemd[1]: Stopping Ceph osd.5 for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:22:08.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:08.504+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:08.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:08.504+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:08.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:08.398+0000 7f59f39f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:22:08.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:08.398+0000 7f59f39f8700 -1 osd.5 795 *** Got signal Terminated *** 2024-02-14T18:22:08.822 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:08.398+0000 7f59f39f8700 -1 osd.5 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:22:09.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:08.999+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:09.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:08 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:08.999+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:09.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:09 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:09.077+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:09.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:09 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:09.077+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:09.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:09 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:09.477+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:09.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:09 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:09.477+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:10.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:10 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:10.021+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:10.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:10 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:10.021+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:10.322 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:10 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:10.083+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:10.322 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:10 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:10.083+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:10.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:10 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:10.512+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:10.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:10 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:10.512+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:11.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:11.002+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:11.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:11.002+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:11.322 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:11.058+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:11.322 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:11.058+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:11.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:11.534+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:11.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:11.534+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:12.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:11.962+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:12.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:11 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:11.962+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:12.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:12.045+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:12.322 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:12.045+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:12.962 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:12.584+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:12.962 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:12.584+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:12.962 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:12.584+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:13.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:13.051+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:13.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:13.051+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:13.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:12.961+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.318026+0000 front 2024-02-14T18:21:33.318194+0000 (oldest deadline 2024-02-14T18:21:56.818067+0000) 2024-02-14T18:22:13.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:12.961+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.318964+0000 front 2024-02-14T18:21:37.319030+0000 (oldest deadline 2024-02-14T18:22:02.618935+0000) 2024-02-14T18:22:13.322 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:12 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5[45387]: 2024-02-14T18:22:12.961+0000 7f59effec700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.820581+0000 front 2024-02-14T18:21:47.820609+0000 (oldest deadline 2024-02-14T18:22:12.420996+0000) 2024-02-14T18:22:13.821 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:13 smithi179 podman[79279]: 2024-02-14 18:22:13.755545958 +0000 UTC m=+5.488099067 container died abc771fd0858adaa28e832a8d50783f9102d3d57d56f3d19db1e4d9b218c39a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-14T18:22:13.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:13.566+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:13.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:13.566+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:13.822 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:13 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:13.566+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:14.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:14 smithi179 podman[79279]: 2024-02-14 18:22:14.216026498 +0000 UTC m=+5.948579593 container cleanup abc771fd0858adaa28e832a8d50783f9102d3d57d56f3d19db1e4d9b218c39a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-02-14T18:22:14.321 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:14 smithi179 bash[79279]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5 2024-02-14T18:22:14.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:14 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:14.044+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:14.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:14 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:14.044+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:14.821 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:14 smithi179 podman[79301]: 2024-02-14 18:22:14.525758646 +0000 UTC m=+0.764909936 container remove abc771fd0858adaa28e832a8d50783f9102d3d57d56f3d19db1e4d9b218c39a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux ) 2024-02-14T18:22:14.822 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:14 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:14.608+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:14.822 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:14 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:14.608+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:14.822 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:14 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:14.608+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:15.154 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:15 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:15.023+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:15.154 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:15 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:15.023+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:15.154 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:15 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:15.023+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:46.738365+0000 front 2024-02-14T18:21:46.738433+0000 (oldest deadline 2024-02-14T18:22:14.938690+0000) 2024-02-14T18:22:15.154 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:14 smithi179 podman[79401]: 2024-02-14 18:22:14.741414916 +0000 UTC m=+0.017995548 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:22:15.154 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:14 smithi179 podman[79401]: 2024-02-14 18:22:14.851554931 +0000 UTC m=+0.128135601 container create 967ecb81994c46ce12c623fe325b2de212b1679ec327b9f9a2c91dfd092432f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-14T18:22:15.155 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:15 smithi179 podman[79401]: 2024-02-14 18:22:15.070504403 +0000 UTC m=+0.347085027 container init 967ecb81994c46ce12c623fe325b2de212b1679ec327b9f9a2c91dfd092432f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:22:15.155 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:15 smithi179 podman[79401]: 2024-02-14 18:22:15.075904985 +0000 UTC m=+0.352485612 container start 967ecb81994c46ce12c623fe325b2de212b1679ec327b9f9a2c91dfd092432f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-14T18:22:15.155 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:15 smithi179 podman[79401]: 2024-02-14 18:22:15.153675846 +0000 UTC m=+0.430256482 container attach 967ecb81994c46ce12c623fe325b2de212b1679ec327b9f9a2c91dfd092432f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-14T18:22:15.568 INFO:journalctl@ceph.osd.5.smithi179.stdout:Feb 14 18:22:15 smithi179 podman[79401]: 2024-02-14 18:22:15.375852537 +0000 UTC m=+0.652433169 container died 967ecb81994c46ce12c623fe325b2de212b1679ec327b9f9a2c91dfd092432f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-14T18:22:15.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:15 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:15.567+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:15.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:15 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:15.567+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:15.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:15 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:15.567+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:16.006 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.5.service' 2024-02-14T18:22:16.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:22:16.048 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-14T18:22:16.049 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-14T18:22:16.049 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.6 2024-02-14T18:22:16.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:16.026+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:16.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:16.026+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:16.322 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:16.026+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:46.738365+0000 front 2024-02-14T18:21:46.738433+0000 (oldest deadline 2024-02-14T18:22:14.938690+0000) 2024-02-14T18:22:16.322 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:16 smithi179 systemd[1]: Stopping Ceph osd.6 for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:22:16.821 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:16.388+0000 7f51a1312700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:22:16.821 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:16.388+0000 7f51a1312700 -1 osd.6 795 *** Got signal Terminated *** 2024-02-14T18:22:16.822 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:16.388+0000 7f51a1312700 -1 osd.6 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:22:16.822 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:16.601+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:16.822 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:16.601+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:16.822 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:16.601+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:16.822 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:16.601+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:17.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:16.996+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:17.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:16.996+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:17.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:16 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:16.996+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:46.738365+0000 front 2024-02-14T18:21:46.738433+0000 (oldest deadline 2024-02-14T18:22:14.938690+0000) 2024-02-14T18:22:17.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:17 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:17.558+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:17.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:17 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:17.558+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:17.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:17 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:17.558+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:17.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:17 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:17.558+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:18.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:17 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:17.970+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:18.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:17 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:17.970+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:18.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:17 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:17.970+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:46.738365+0000 front 2024-02-14T18:21:46.738433+0000 (oldest deadline 2024-02-14T18:22:14.938690+0000) 2024-02-14T18:22:18.936 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:18 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:18.578+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:18.936 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:18 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:18.578+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:18.936 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:18 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:18.578+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:18.936 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:18 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:18.578+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:19.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:18 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:18.934+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:19.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:18 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:18.934+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:19.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:18 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:18.934+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:46.738365+0000 front 2024-02-14T18:21:46.738433+0000 (oldest deadline 2024-02-14T18:22:14.938690+0000) 2024-02-14T18:22:19.928 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:19 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:19.572+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:19.929 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:19 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:19.572+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:19.929 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:19 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:19.572+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:19.929 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:19 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:19.572+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:20.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:19 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:19.927+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:20.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:19 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:19.927+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:20.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:19 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:19.927+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:46.738365+0000 front 2024-02-14T18:21:46.738433+0000 (oldest deadline 2024-02-14T18:22:14.938690+0000) 2024-02-14T18:22:20.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:20.539+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:20.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:20.539+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:20.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:20.539+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:20.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:20.540+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:21.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:20.948+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:33.336748+0000 front 2024-02-14T18:21:33.336600+0000 (oldest deadline 2024-02-14T18:21:58.636578+0000) 2024-02-14T18:22:21.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:20.948+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:40.937639+0000 front 2024-02-14T18:21:40.937686+0000 (oldest deadline 2024-02-14T18:22:02.037388+0000) 2024-02-14T18:22:21.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:20 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6[51338]: 2024-02-14T18:22:20.948+0000 7f51998fe700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:46.738365+0000 front 2024-02-14T18:21:46.738433+0000 (oldest deadline 2024-02-14T18:22:14.938690+0000) 2024-02-14T18:22:21.821 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:21 smithi179 podman[79534]: 2024-02-14 18:22:21.737414293 +0000 UTC m=+5.474348293 container died 35248f8df8eb3b796e898b9b935f3500c2dab9d5601a2eee5e2863f631360457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6, org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True) 2024-02-14T18:22:21.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:21.501+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:21.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:21.501+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:21.822 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:21.501+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:21.822 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:21 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:21.501+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:22.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:22 smithi179 podman[79534]: 2024-02-14 18:22:22.084668768 +0000 UTC m=+5.821602767 container cleanup 35248f8df8eb3b796e898b9b935f3500c2dab9d5601a2eee5e2863f631360457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-14T18:22:22.321 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:22 smithi179 bash[79534]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6 2024-02-14T18:22:22.693 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:22 smithi179 podman[79556]: 2024-02-14 18:22:22.369244792 +0000 UTC m=+0.631246245 container remove 35248f8df8eb3b796e898b9b935f3500c2dab9d5601a2eee5e2863f631360457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T18:22:22.693 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:22.502+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:22.693 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:22.502+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:22.693 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:22.502+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:22.694 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:22 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:22.502+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:22.990 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:22 smithi179 podman[79652]: 2024-02-14 18:22:22.595407881 +0000 UTC m=+0.018802937 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:22:22.990 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:22 smithi179 podman[79652]: 2024-02-14 18:22:22.703418196 +0000 UTC m=+0.126813250 container create fc72abca8b5a69a86ab6fe38113c73bf3ae2d916378f7ec110d3bceaed1507e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-14T18:22:22.990 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:22 smithi179 podman[79652]: 2024-02-14 18:22:22.923310371 +0000 UTC m=+0.346705431 container init fc72abca8b5a69a86ab6fe38113c73bf3ae2d916378f7ec110d3bceaed1507e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-14T18:22:22.990 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:22 smithi179 podman[79652]: 2024-02-14 18:22:22.92932686 +0000 UTC m=+0.352721919 container start fc72abca8b5a69a86ab6fe38113c73bf3ae2d916378f7ec110d3bceaed1507e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:22:22.990 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:22 smithi179 podman[79652]: 2024-02-14 18:22:22.989292471 +0000 UTC m=+0.412687534 container attach fc72abca8b5a69a86ab6fe38113c73bf3ae2d916378f7ec110d3bceaed1507e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-14T18:22:23.509 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:23 smithi179 podman[79699]: 2024-02-14 18:22:23.245185819 +0000 UTC m=+0.018916735 container died fc72abca8b5a69a86ab6fe38113c73bf3ae2d916378f7ec110d3bceaed1507e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:22:23.510 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:23.489+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:23.510 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:23.489+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:23.510 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:23.489+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:23.510 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:23.489+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-02-14T18:21:57.504290+0000 front 2024-02-14T18:21:57.504200+0000 (oldest deadline 2024-02-14T18:22:23.404220+0000) 2024-02-14T18:22:23.510 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:23 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:23.489+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:23.773 INFO:journalctl@ceph.osd.6.smithi179.stdout:Feb 14 18:22:23 smithi179 podman[79699]: 2024-02-14 18:22:23.75075604 +0000 UTC m=+0.524486952 container remove fc72abca8b5a69a86ab6fe38113c73bf3ae2d916378f7ec110d3bceaed1507e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5) 2024-02-14T18:22:23.778 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.6.service' 2024-02-14T18:22:23.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:22:23.819 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-14T18:22:23.819 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-14T18:22:23.819 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.7 2024-02-14T18:22:24.071 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:23 smithi179 systemd[1]: Stopping Ceph osd.7 for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:22:24.476 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:24.164+0000 7fce47f6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:22:24.476 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:24.164+0000 7fce47f6d700 -1 osd.7 795 *** Got signal Terminated *** 2024-02-14T18:22:24.476 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:24.164+0000 7fce47f6d700 -1 osd.7 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-14T18:22:24.476 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:24.474+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:24.476 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:24.474+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:24.476 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:24.474+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:24.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:24.474+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-02-14T18:21:57.504290+0000 front 2024-02-14T18:21:57.504200+0000 (oldest deadline 2024-02-14T18:22:23.404220+0000) 2024-02-14T18:22:24.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:24 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:24.474+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:25.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:25 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:25.454+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:25.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:25 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:25.454+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:25.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:25 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:25.454+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:25.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:25 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:25.454+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-02-14T18:21:57.504290+0000 front 2024-02-14T18:21:57.504200+0000 (oldest deadline 2024-02-14T18:22:23.404220+0000) 2024-02-14T18:22:25.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:25 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:25.454+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:26.823 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:26 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:26.422+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:26.823 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:26 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:26.422+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:26.823 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:26 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:26.422+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:26.823 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:26 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:26.422+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-02-14T18:21:57.504290+0000 front 2024-02-14T18:21:57.504200+0000 (oldest deadline 2024-02-14T18:22:23.404220+0000) 2024-02-14T18:22:26.824 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:26 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:26.422+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:27.722 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:27 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:27.374+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:27.722 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:27 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:27.374+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:27.722 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:27 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:27.374+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:27.723 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:27 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:27.374+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-02-14T18:21:57.504290+0000 front 2024-02-14T18:21:57.504200+0000 (oldest deadline 2024-02-14T18:22:23.404220+0000) 2024-02-14T18:22:27.723 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:27 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:27.374+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:28.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:28 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:28.346+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6806 osd.0 since back 2024-02-14T18:21:31.900004+0000 front 2024-02-14T18:21:31.900147+0000 (oldest deadline 2024-02-14T18:21:57.100575+0000) 2024-02-14T18:22:28.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:28 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:28.346+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6814 osd.1 since back 2024-02-14T18:21:37.101219+0000 front 2024-02-14T18:21:37.101119+0000 (oldest deadline 2024-02-14T18:22:02.401023+0000) 2024-02-14T18:22:28.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:28 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:28.346+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6822 osd.2 since back 2024-02-14T18:21:47.503211+0000 front 2024-02-14T18:21:47.503195+0000 (oldest deadline 2024-02-14T18:22:12.203110+0000) 2024-02-14T18:22:28.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:28 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:28.346+0000 7fce40559700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.100:6830 osd.3 since back 2024-02-14T18:21:57.504290+0000 front 2024-02-14T18:21:57.504200+0000 (oldest deadline 2024-02-14T18:22:23.404220+0000) 2024-02-14T18:22:28.821 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:28 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7[57455]: 2024-02-14T18:22:28.346+0000 7fce40559700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24397.0:2191 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-14T18:22:29.928 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:29 smithi179 podman[79783]: 2024-02-14 18:22:29.609539498 +0000 UTC m=+5.571343678 container died edc4cc9a4659f58e0da00dd4e2a595565be4ceb3337a5cf62bc4a23df2808da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-14T18:22:30.321 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:29 smithi179 podman[79783]: 2024-02-14 18:22:29.928168026 +0000 UTC m=+5.889972194 container cleanup edc4cc9a4659f58e0da00dd4e2a595565be4ceb3337a5cf62bc4a23df2808da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T18:22:30.321 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:29 smithi179 bash[79783]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7 2024-02-14T18:22:30.322 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:30 smithi179 podman[79817]: 2024-02-14 18:22:30.104239635 +0000 UTC m=+0.494683995 container remove edc4cc9a4659f58e0da00dd4e2a595565be4ceb3337a5cf62bc4a23df2808da3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-14T18:22:30.799 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:30 smithi179 podman[79912]: 2024-02-14 18:22:30.400588645 +0000 UTC m=+0.018854308 image pull 170846cf6432fa177c159b5d8a38454ab2459d562a59c58b9aa838f6e31ddf86 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b 2024-02-14T18:22:30.799 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:30 smithi179 podman[79912]: 2024-02-14 18:22:30.52204873 +0000 UTC m=+0.140314430 container create 21e71486f59576153c89882a9038b1a5db64418754380dc75cedee68c2008460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T18:22:30.800 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:30 smithi179 podman[79912]: 2024-02-14 18:22:30.715944107 +0000 UTC m=+0.334209764 container init 21e71486f59576153c89882a9038b1a5db64418754380dc75cedee68c2008460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-14T18:22:30.800 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:30 smithi179 podman[79912]: 2024-02-14 18:22:30.721332813 +0000 UTC m=+0.339598467 container start 21e71486f59576153c89882a9038b1a5db64418754380dc75cedee68c2008460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-14T18:22:30.800 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:30 smithi179 podman[79912]: 2024-02-14 18:22:30.798955567 +0000 UTC m=+0.417221229 container attach 21e71486f59576153c89882a9038b1a5db64418754380dc75cedee68c2008460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-14T18:22:31.071 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:31 smithi179 conmon[79928]: conmon 21e71486f59576153c89 : Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-21e71486f59576153c89882a9038b1a5db64418754380dc75cedee68c2008460.scope/memory.events 2024-02-14T18:22:31.071 INFO:journalctl@ceph.osd.7.smithi179.stdout:Feb 14 18:22:31 smithi179 podman[79912]: 2024-02-14 18:22:31.014247405 +0000 UTC m=+0.632513063 container died 21e71486f59576153c89882a9038b1a5db64418754380dc75cedee68c2008460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-osd-7-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9) 2024-02-14T18:22:31.595 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@osd.7.service' 2024-02-14T18:22:31.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:22:31.636 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-14T18:22:31.637 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-14T18:22:31.637 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@rgw.foo.a 2024-02-14T18:22:31.934 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 18:22:31 smithi100 systemd[1]: Stopping Ceph rgw.foo.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:22:32.399 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 18:22:31 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-rgw-foo-a[67771]: 2024-02-14T18:22:31.947+0000 7f36ec882700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-14T18:22:32.399 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 18:22:31 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-rgw-foo-a[67771]: 2024-02-14T18:22:31.947+0000 7f36fb0f3ac0 -1 shutting down 2024-02-14T18:22:42.312 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 18:22:41 smithi100 bash[159458]: time="2024-02-14T18:22:41Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-14T18:22:42.312 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 18:22:41 smithi100 podman[159458]: 2024-02-14 18:22:41.976058091 +0000 UTC m=+10.147370643 container died 63622df17c34d3d9d3f8b0de18dbad67d1e4ac9e60324c05f84404b24b7b2007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T18:22:42.650 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 18:22:42 smithi100 podman[159458]: 2024-02-14 18:22:42.410204592 +0000 UTC m=+10.581517125 container cleanup 63622df17c34d3d9d3f8b0de18dbad67d1e4ac9e60324c05f84404b24b7b2007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-14T18:22:42.650 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 18:22:42 smithi100 bash[159458]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-rgw-foo-a 2024-02-14T18:22:42.971 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 18:22:42 smithi100 podman[159496]: 2024-02-14 18:22:42.69306422 +0000 UTC m=+0.712895434 container remove 63622df17c34d3d9d3f8b0de18dbad67d1e4ac9e60324c05f84404b24b7b2007 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-14T18:22:42.971 INFO:journalctl@ceph.rgw.foo.a.smithi100.stdout:Feb 14 18:22:42 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-02-14T18:22:42.988 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@rgw.foo.a.service' 2024-02-14T18:22:43.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:22:43.027 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-14T18:22:43.027 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-14T18:22:43.027 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@prometheus.a 2024-02-14T18:22:43.315 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 systemd[1]: Stopping Ceph prometheus.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:22:43.571 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.315Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-14T18:22:43.571 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.315Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-14T18:22:43.571 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.315Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-14T18:22:43.571 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.315Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-14T18:22:43.571 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.315Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-14T18:22:43.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.315Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-14T18:22:43.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.315Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-14T18:22:43.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.315Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-14T18:22:43.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.315Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-14T18:22:43.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.317Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-14T18:22:43.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.317Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-14T18:22:43.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a[67838]: ts=2024-02-14T18:22:43.317Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-14T18:22:43.572 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 podman[80045]: 2024-02-14 18:22:43.325763049 +0000 UTC m=+0.105034325 container died 75209456d1d62cbf131e00a319b7c4241a664d5365f9e54f9b11767fd1395a7b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T18:22:43.855 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 podman[80045]: 2024-02-14 18:22:43.662092815 +0000 UTC m=+0.441364085 container cleanup 75209456d1d62cbf131e00a319b7c4241a664d5365f9e54f9b11767fd1395a7b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-14T18:22:43.856 INFO:journalctl@ceph.prometheus.a.smithi179.stdout:Feb 14 18:22:43 smithi179 bash[80045]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-prometheus-a 2024-02-14T18:22:44.073 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@prometheus.a.service' 2024-02-14T18:22:44.110 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-14T18:22:44.111 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-14T18:22:44.111 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-14T18:22:44.305 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T18:22:46.110 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 18:22:45 smithi100 systemd[1]: Stopping Ceph alertmanager.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:22:46.399 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 18:22:46 smithi100 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a[78809]: ts=2024-02-14T18:22:46.109Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-14T18:22:46.399 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 18:22:46 smithi100 podman[159980]: 2024-02-14 18:22:46.150994392 +0000 UTC m=+0.144038144 container died c2cd92038c03fe377d2a5553355dbfdfdffb83f94314ca6cfbf39bc2738a8c88 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T18:22:46.758 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 18:22:46 smithi100 podman[159980]: 2024-02-14 18:22:46.515403675 +0000 UTC m=+0.508447433 container cleanup c2cd92038c03fe377d2a5553355dbfdfdffb83f94314ca6cfbf39bc2738a8c88 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T18:22:46.758 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 18:22:46 smithi100 bash[159980]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a 2024-02-14T18:22:47.149 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 18:22:46 smithi100 podman[159999]: 2024-02-14 18:22:46.758235204 +0000 UTC m=+0.603148043 container remove c2cd92038c03fe377d2a5553355dbfdfdffb83f94314ca6cfbf39bc2738a8c88 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-14T18:22:47.149 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 18:22:46 smithi100 podman[159999]: 2024-02-14 18:22:46.833919631 +0000 UTC m=+0.678832421 volume remove 87e963d41ee93f6ba3b90b3820ebeb62639fcd52dc8e88c39a913b1d7b6cb865 2024-02-14T18:22:47.149 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 18:22:46 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-14T18:22:47.149 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 18:22:46 smithi100 systemd[1]: Stopped Ceph alertmanager.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:22:47.150 INFO:journalctl@ceph.alertmanager.a.smithi100.stdout:Feb 14 18:22:46 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@alertmanager.a.service: Consumed 2.428s CPU time. 2024-02-14T18:22:47.649 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 18:22:47 smithi100 systemd[1]: Stopping Ceph node-exporter.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:22:47.649 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 18:22:47 smithi100 podman[160117]: 2024-02-14 18:22:47.408046127 +0000 UTC m=+0.093031409 container died 7eabaaf7305d5ac2036bd05d074e84b2944a040542f605a8d0873cfeb7dc073e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T18:22:48.062 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 18:22:47 smithi100 podman[160117]: 2024-02-14 18:22:47.761000191 +0000 UTC m=+0.445985475 container cleanup 7eabaaf7305d5ac2036bd05d074e84b2944a040542f605a8d0873cfeb7dc073e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T18:22:48.062 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 18:22:47 smithi100 bash[160117]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a 2024-02-14T18:22:48.371 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 18:22:48 smithi100 podman[160135]: 2024-02-14 18:22:48.06217369 +0000 UTC m=+0.655016122 container remove 7eabaaf7305d5ac2036bd05d074e84b2944a040542f605a8d0873cfeb7dc073e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-14T18:22:48.371 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 18:22:48 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-14T18:22:48.371 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 18:22:48 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-14T18:22:48.371 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 18:22:48 smithi100 systemd[1]: Stopped Ceph node-exporter.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:22:48.372 INFO:journalctl@ceph.node-exporter.a.smithi100.stdout:Feb 14 18:22:48 smithi100 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@node-exporter.a.service: Consumed 5.207s CPU time. 2024-02-14T18:22:48.843 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-14T18:22:49.036 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T18:22:50.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:22:50 smithi179 systemd[1]: Stopping Ceph iscsi.iscsi.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:22:50.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:22:50 smithi179 ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a[62586]: debug Shutdown received 2024-02-14T18:23:00.784 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:23:00 smithi179 bash[80466]: time="2024-02-14T18:23:00Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-02-14T18:23:00.784 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:23:00 smithi179 podman[80466]: 2024-02-14 18:23:00.512410246 +0000 UTC m=+10.127706263 container died a7ccf93cf909e2dcf07c072be15c555c9a5ac3e6bb3b08139ab723be3e7aadb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-14T18:23:01.040 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:23:00 smithi179 podman[80466]: 2024-02-14 18:23:00.940446646 +0000 UTC m=+10.555742667 container cleanup a7ccf93cf909e2dcf07c072be15c555c9a5ac3e6bb3b08139ab723be3e7aadb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-14T18:23:01.041 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:23:00 smithi179 bash[80466]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a 2024-02-14T18:23:01.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:23:01 smithi179 podman[80491]: 2024-02-14 18:23:01.151747525 +0000 UTC m=+0.637802753 container remove a7ccf93cf909e2dcf07c072be15c555c9a5ac3e6bb3b08139ab723be3e7aadb9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e74cf8e1e43e8dfa713eb599fa0104396b2e29af34a3f40d868498c8111bdb4b, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-iscsi-iscsi-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-02-14-1033-8727ce9, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-14T18:23:01.321 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:23:01 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-14T18:23:01.692 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:23:01 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-14T18:23:01.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:23:01 smithi179 systemd[1]: Stopped Ceph iscsi.iscsi.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:23:01.693 INFO:journalctl@ceph.iscsi.iscsi.a.smithi179.stdout:Feb 14 18:23:01 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Consumed 4.354s CPU time. 2024-02-14T18:23:03.044 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:23:02 smithi179 systemd[1]: Stopping Ceph grafana.a for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:23:03.307 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:23:03 smithi179 podman[80779]: 2024-02-14 18:23:03.044352218 +0000 UTC m=+0.123011474 container died a4c562e237e55df4a9ec4d5087751c691061da7f0901fb723c6883b5c2d29639 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, description=Ceph Grafana Container, io.buildah.version=1.32.0, name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container, distribution-scope=public, io.openshift.tags=base rhel8, io.openshift.expose-services=, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc.) 2024-02-14T18:23:03.571 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:23:03 smithi179 podman[80779]: 2024-02-14 18:23:03.380333532 +0000 UTC m=+0.458992800 container cleanup a4c562e237e55df4a9ec4d5087751c691061da7f0901fb723c6883b5c2d29639 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, io.buildah.version=1.32.0, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., distribution-scope=public, architecture=x86_64, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, name=ubi8, version=8.5) 2024-02-14T18:23:03.571 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:23:03 smithi179 bash[80779]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a 2024-02-14T18:23:03.571 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:23:03 smithi179 podman[80798]: 2024-02-14 18:23:03.522994086 +0000 UTC m=+0.479576477 container remove a4c562e237e55df4a9ec4d5087751c691061da7f0901fb723c6883b5c2d29639 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-grafana-a, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, architecture=x86_64, io.openshift.expose-services=, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, name=ubi8, maintainer=Paul Cuzner , vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, distribution-scope=public, io.buildah.version=1.32.0, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., release=239.1651231664) 2024-02-14T18:23:03.571 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:23:03 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-14T18:23:04.071 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 18:23:03 smithi179 systemd[1]: Stopping Ceph node-exporter.b for d69a9998-cb5f-11ee-95b9-87774f69a715... 2024-02-14T18:23:04.071 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:23:03 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-14T18:23:04.071 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:23:03 smithi179 systemd[1]: Stopped Ceph grafana.a for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:23:04.071 INFO:journalctl@ceph.grafana.a.smithi179.stdout:Feb 14 18:23:03 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@grafana.a.service: Consumed 18.046s CPU time. 2024-02-14T18:23:04.476 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 18:23:04 smithi179 podman[80914]: 2024-02-14 18:23:04.147966072 +0000 UTC m=+0.082199498 container died 1c0b8ee2fef23ef74f79d72c54a060004eb96a1340283f4b90203d469f86ac38 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T18:23:04.476 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 18:23:04 smithi179 podman[80914]: 2024-02-14 18:23:04.475546571 +0000 UTC m=+0.409780000 container cleanup 1c0b8ee2fef23ef74f79d72c54a060004eb96a1340283f4b90203d469f86ac38 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T18:23:04.821 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 18:23:04 smithi179 bash[80914]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b 2024-02-14T18:23:04.821 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 18:23:04 smithi179 podman[80933]: 2024-02-14 18:23:04.6768415 +0000 UTC m=+0.529791990 container remove 1c0b8ee2fef23ef74f79d72c54a060004eb96a1340283f4b90203d469f86ac38 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d69a9998-cb5f-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-14T18:23:04.821 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 18:23:04 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-14T18:23:05.308 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 18:23:04 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-14T18:23:05.308 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 18:23:04 smithi179 systemd[1]: Stopped Ceph node-exporter.b for d69a9998-cb5f-11ee-95b9-87774f69a715. 2024-02-14T18:23:05.308 INFO:journalctl@ceph.node-exporter.b.smithi179.stdout:Feb 14 18:23:04 smithi179 systemd[1]: ceph-d69a9998-cb5f-11ee-95b9-87774f69a715@node-exporter.b.service: Consumed 5.127s CPU time. 2024-02-14T18:23:05.499 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T18:23:05.535 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-14T18:23:05.565 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-14T18:23:05.567 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815/remote/smithi100/crash 2024-02-14T18:23:05.568 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/crash -- . 2024-02-14T18:23:05.624 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-14T18:23:05.624 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-02-14T18:23:05.626 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815/remote/smithi179/crash 2024-02-14T18:23:05.627 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/crash -- . 2024-02-14T18:23:05.666 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-14T18:23:05.666 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2024-02-14T18:23:05.667 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-14T18:23:05.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-14T18:23:05.701 INFO:teuthology.orchestra.run.smithi100.stdout:2024-02-14T17:50:00.000169+0000 mon.a (mon.0) 1623 : cluster 3 [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-14T18:23:05.701 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-14T18:23:05.701 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[SEC\]' /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-14T18:23:05.773 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]' /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-14T18:23:05.845 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[WRN\]' /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-14T18:23:05.916 INFO:teuthology.orchestra.run.smithi100.stdout:2024-02-14T17:50:00.000169+0000 mon.a (mon.0) 1623 : cluster 3 [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-14T18:23:05.916 INFO:tasks.cephadm:Compressing logs... 2024-02-14T18:23:05.916 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-14T18:23:05.958 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-14T18:23:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-14T18:23:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:: No such file or directory 2024-02-14T18:23:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mon.a.log 2024-02-14T18:23:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-14T18:23:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-14T18:23:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mon.a.log: /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.1.log 2024-02-14T18:23:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.3.log 2024-02-14T18:23:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log: /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mgr.y.log 2024-02-14T18:23:05.984 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.0.log 2024-02-14T18:23:05.986 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-14T18:23:05.986 INFO:teuthology.orchestra.run.smithi179.stderr:: No such file or directory 2024-02-14T18:23:05.986 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.6.log 2024-02-14T18:23:05.986 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-14T18:23:05.986 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.4.log 2024-02-14T18:23:05.990 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.6.log: /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.7.log 2024-02-14T18:23:05.991 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.5.log 2024-02-14T18:23:05.991 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.4.log: /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mgr.x.log 2024-02-14T18:23:05.991 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/tcmu-runner.log 2024-02-14T18:23:05.991 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-14T18:23:05.991 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/tcmu-runner.log: 67.8% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/tcmu-runner.log.gz 2024-02-14T18:23:05.991 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.log 2024-02-14T18:23:05.991 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log: 81.6% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-14T18:23:05.991 INFO:teuthology.orchestra.run.smithi179.stderr: 85.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-14T18:23:05.991 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-14T18:23:05.992 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mgr.y.log: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-14T18:23:05.993 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-14T18:23:05.995 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.log: 87.9% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-14T18:23:05.997 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mon.b.log 2024-02-14T18:23:05.997 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-volume.log: 92.3%/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mon.b.log: -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-14T18:23:06.000 INFO:teuthology.orchestra.run.smithi179.stderr: 92.7% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mgr.x.log.gz 2024-02-14T18:23:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.0.log: 95.2% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-14T18:23:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.log 2024-02-14T18:23:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log: 88.9% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-14T18:23:06.005 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.2.log 2024-02-14T18:23:06.011 INFO:teuthology.orchestra.run.smithi179.stderr: 91.0% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-14T18:23:06.014 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.log: 93.3%gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-14T18:23:06.016 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.2.log: -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-14T18:23:06.022 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mon.c.log 2024-02-14T18:23:06.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-volume.log: /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mon.c.log: 94.3% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-14T18:23:06.060 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-14T18:23:06.568 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mgr.y.log.gz 2024-02-14T18:23:07.462 INFO:teuthology.orchestra.run.smithi179.stderr: 93.0% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mon.b.log.gz 2024-02-14T18:23:07.714 INFO:teuthology.orchestra.run.smithi100.stderr: 92.6% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mon.c.log.gz 2024-02-14T18:23:08.741 INFO:teuthology.orchestra.run.smithi100.stderr: 92.0% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-mon.a.log.gz 2024-02-14T18:23:14.261 INFO:teuthology.orchestra.run.smithi179.stderr: 93.9% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.6.log.gz 2024-02-14T18:23:14.716 INFO:teuthology.orchestra.run.smithi179.stderr: 93.9% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.7.log.gz 2024-02-14T18:23:14.769 INFO:teuthology.orchestra.run.smithi179.stderr: 93.9% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.5.log.gz 2024-02-14T18:23:14.930 INFO:teuthology.orchestra.run.smithi100.stderr: 94.0% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.3.log.gz 2024-02-14T18:23:15.027 INFO:teuthology.orchestra.run.smithi100.stderr: 94.0% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.0.log.gz 2024-02-14T18:23:15.058 INFO:teuthology.orchestra.run.smithi179.stderr: 94.0% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.4.log.gz 2024-02-14T18:23:15.061 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-02-14T18:23:15.061 INFO:teuthology.orchestra.run.smithi179.stderr:real 0m9.088s 2024-02-14T18:23:15.061 INFO:teuthology.orchestra.run.smithi179.stderr:user 0m35.065s 2024-02-14T18:23:15.061 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m1.273s 2024-02-14T18:23:15.514 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.2.log.gz 2024-02-14T18:23:15.730 INFO:teuthology.orchestra.run.smithi100.stderr: 94.0% -- replaced with /var/log/ceph/d69a9998-cb5f-11ee-95b9-87774f69a715/ceph-osd.1.log.gz 2024-02-14T18:23:15.732 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-02-14T18:23:15.732 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m9.763s 2024-02-14T18:23:15.732 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m40.091s 2024-02-14T18:23:15.732 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m1.396s 2024-02-14T18:23:15.733 INFO:tasks.cephadm:Archiving logs... 2024-02-14T18:23:15.733 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815/remote/smithi100/log 2024-02-14T18:23:15.734 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-14T18:23:21.162 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815/remote/smithi179/log 2024-02-14T18:23:21.162 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-14T18:23:26.131 INFO:tasks.cephadm:Removing cluster... 2024-02-14T18:23:26.132 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 --force 2024-02-14T18:23:26.324 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T18:23:26.682 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d69a9998-cb5f-11ee-95b9-87774f69a715 --force 2024-02-14T18:23:26.873 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: d69a9998-cb5f-11ee-95b9-87774f69a715 2024-02-14T18:23:27.166 INFO:tasks.cephadm:Removing cephadm ... 2024-02-14T18:23:27.166 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-14T18:23:27.182 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-14T18:23:27.197 INFO:tasks.cephadm:Teardown complete 2024-02-14T18:23:27.197 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-14T18:23:27.207 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-14T18:23:27.207 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-14T18:23:27.224 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-14T18:23:27.316 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-14T18:23:27.326 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-14T18:23:27.326 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T18:23:27.328 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-14T18:23:27.340 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-02-14T18:23:27.342 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-02-14T18:23:27.344 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T18:23:27.344 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-02-14T18:23:27.344 INFO:teuthology.orchestra.run.smithi100.stdout:^- hv01.front.sepia.ceph.com 2 6 377 34 +2714us[+2708us] +/- 130ms 2024-02-14T18:23:27.344 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 +197us[ +191us] +/- 62ms 2024-02-14T18:23:27.344 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -265us[ -265us] +/- 71ms 2024-02-14T18:23:27.344 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T18:23:27.345 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-14T18:23:27.346 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-02-14T18:23:27.346 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +2177us[+2188us] +/- 130ms 2024-02-14T18:23:27.346 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv02.front.sepia.ceph.com 3 6 377 33 -332us[ -320us] +/- 62ms 2024-02-14T18:23:27.346 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -793us[ -781us] +/- 71ms 2024-02-14T18:23:27.346 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-14T18:23:27.347 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-14T18:23:27.355 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-14T18:23:27.356 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-14T18:23:27.386 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-14T18:23:27.466 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-14T18:23:27.543 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T18:23:27.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T18:23:27.573 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-14T18:23:27.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T18:23:27.603 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-14T18:23:27.612 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-14T18:23:27.636 INFO:teuthology.task.internal:Duration was 3148.689182 seconds 2024-02-14T18:23:27.637 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-14T18:23:27.670 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-14T18:23:27.670 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-14T18:23:27.672 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-14T18:23:27.712 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T18:23:27.715 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-14T18:23:27.939 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-14T18:23:27.939 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-02-14T18:23:27.939 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-14T18:23:27.998 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-02-14T18:23:27.998 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-14T18:23:28.018 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-14T18:23:28.018 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-14T18:23:28.041 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-14T18:23:28.086 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-14T18:23:28.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-14T18:23:28.110 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-14T18:23:28.560 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-14T18:23:28.571 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-14T18:23:28.571 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-14T18:23:28.597 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-14T18:23:28.624 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-14T18:23:28.711 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-14T18:23:28.713 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-14T18:23:28.736 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-02-14T18:23:28.739 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-02-14T18:23:28.765 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-14T18:23:28.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T18:23:28.806 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-14T18:23:28.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-14T18:23:28.821 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-14T18:23:29.035 INFO:teuthology.task.internal:Transferring archived files... 2024-02-14T18:23:29.036 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815/remote/smithi100 2024-02-14T18:23:29.036 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-14T18:23:29.099 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815/remote/smithi179 2024-02-14T18:23:29.099 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-14T18:23:29.139 INFO:teuthology.task.internal:Removing archive directory... 2024-02-14T18:23:29.139 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-14T18:23:29.141 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-14T18:23:29.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-14T18:23:29.198 INFO:teuthology.task.internal:Not uploading archives. 2024-02-14T18:23:29.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-14T18:23:29.229 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-14T18:23:29.229 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-14T18:23:29.231 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-14T18:23:29.243 INFO:teuthology.orchestra.run.smithi100.stdout: 266030 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 14 18:23 /home/ubuntu/cephtest 2024-02-14T18:23:29.246 INFO:teuthology.orchestra.run.smithi179.stdout: 265293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 14 18:23 /home/ubuntu/cephtest 2024-02-14T18:23:29.247 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-14T18:23:29.263 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-14T18:23:29.329 INFO:teuthology.nuke:Checking targets against current locks 2024-02-14T18:23:29.353 DEBUG:teuthology.nuke:shortname: smithi100 2024-02-14T18:23:29.354 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T18:23:29.378 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 17:22:38.396018', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEqHc+axRfi/36fqFGGPS8jxmYiw1/UCCGtjBUSvJqo13qD9za4uRqHlUHPIR4RWqKOA74POHXdpLiQSSsIra4='} 2024-02-14T18:23:29.381 DEBUG:teuthology.nuke:shortname: smithi179 2024-02-14T18:23:29.381 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T18:23:29.405 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-14_13:54:10-orch:cephadm-wip-guits-testing-2024-02-14-1033-distro-default-smithi/7559815', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 17:22:38.397865', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEFGDxPeEqtQJ9xNWG6du0Xh67LzpgHU8T1ihKTxxuqMTYU/2lVxGA3qH4l29cZMSeA0WXqZ75rdt9rTVkCwL78='} 2024-02-14T18:23:29.430 INFO:teuthology.orchestra.console.smithi100:Power off 2024-02-14T18:23:29.430 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T18:23:29.455 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-02-14T18:23:29.455 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:23:29.471 INFO:teuthology.orchestra.console.smithi179:Power off 2024-02-14T18:23:29.471 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T18:23:29.480 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-02-14T18:23:29.493 DEBUG:teuthology.orchestra.console.smithi179:power off output: Chassis Power Control: Down/Off 2024-02-14T18:23:29.493 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:23:29.516 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-02-14T18:23:33.481 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:23:33.517 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:23:33.591 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-02-14T18:23:33.625 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-02-14T18:23:37.597 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:23:37.626 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:23:37.707 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-02-14T18:23:37.736 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-02-14T18:23:37.808 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-02-14T18:23:41.743 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:23:41.853 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off 2024-02-14T18:23:41.954 INFO:teuthology.orchestra.console.smithi179:Power off completed 2024-02-14T18:23:42.058 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} duration: 3148.6891820430756 failure_reason: '"2024-02-14T17:50:00.000169+0000 mon.a (mon.0) 1623 : cluster 3 [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets" in cluster log' flavor: default owner: scheduled_gabrioux@teuthology success: false 2024-02-14T18:23:42.058 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T18:23:42.175 INFO:teuthology.run:FAIL